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FOREWORD 



Whan the International Union of Pure and Applied chemistry 
(IUPAC) established in 1965 a Committee on the Teaching of Chemistry, 
one of the first tasks of this Committee was to review major areas of 
chemical education and decide on priorities for action. It was 
quickly agreed that examiriations in chemistry play a major part in 
determining how the subject is taught and that if new approaches to 
teaching chemistry were desired, then one way of bringing this scout 
would be to change the type of questions asked in public and internal 
eliminations. 

Accordingly the IUPAC Committee on the Teaching of Chemistry 
invited Mr. J.C. Mathewe (U.K.) to prepare a report on "The Effect, of 
Examinations in Determining Chemistry Curricula up to University Level 11 
vrfiich was published in Pure and Applied Chemistry by But ter worths in 
1967. This Report was mainly a comparative survey of selected eoun^ 
tries in Europe together with U.S.A* and Japan. 

Subsequently a report was submitted to the IUPAC Committee on 
similar problems viewed from the practice in France and countries in 
Southern Europe, whose educational systems and examination procedures 
are somewhat different from Western Europe arid the 0 . S * A * This report 
has not yet been published * 

The Report that follows in this booklet describes the proceedings 
and conclxisions of an international workshop held in Ceylon in August, 
196s, which was convened by IUPAC as a foUow-up to the original 
Mathews Report. The Workshop was imder the Chairmanship of Professor 
H.F. Halliwell (U.K,), who has had wide experience in chemical educa- 
tion throughout the world, and the Secretary to the Workshop was Mr. D*G. 
Chisman, Secretary of IUPAC Committee on the Teaching of Chemistry* 

Mr, J»C* Mathews himself was among the ten participants from outside 
Ceylon. 

The Workshop was held in Ceylon at the invitation of the Ceylonese 
Ministry of Education and was timed to follow a seminar oh evaluation 
and curriculum development sponsored by the Ministry of Education in 
Ceylon, the British Ministry of Overseas Development arai British Council 
which was under the direction of Mr. J*C. Mathews. The IUPAC Committee 
on the Teaching of Chemistry were pleased to have this opportunity of 
arranging a Workshop in Ceylon on this topic since the progress made by 
the Ceylon Ministry of Education in revising chemistry curricula and 
changing tha whole approach to examining ir* chemistry is very considerable 
indeed. They were also pleased to be able to collaborate with UNESCO in 
the joint sponsorship of the Workshop on this occasion since UNESCO Ms 
Chemistry Pilot Project in Asia has been involved with similar problems 
of evaluation. In particular the UNESCO Study Group in Ceylon, associated 
with the Pilot Project was involved in the Workshop. 
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INTRODUCTION 



In order to communicate clearly and effectively in any 
systematic activity it becomes necessary to develop concepts 
and at the same time to evolve symbolism and technical terms 
which carry specific and unambiguous meaning* 



Chemistry as a subject provides a good example of the 
need to develop such concepts with associated symbolism© and 
technical terminology. The emergence of chemical , symbolic 
nomenclature in the early 19th century contributed signifi- 
cantly to the growth of chemistry* In recent years, the 
study of organic reaction mechanism and the concepts. evolved 
for its interpretation and communication provides a further 
example * 

In the field of curriculum development and assessment 
the use of specialised terminology and symbolism, such as 
content objectives, behavioural objectives, tables of speci- 
fications, reliability coefficient, difficulty indices and 
discrimination coefficients, illustrate an analogous phase of 
professional activity* . . 

In this report terms that are commonly used in curricu- 
lum development have been included, and there is a glossary 
of such terms for those new to this field,. It is suggested, 
however, that those chemists and administrators who are now 
engaged in this important area of evaluation and assessment 
of chemists curricula should become sensitive to the concepts 
involved and familiar in the use of related symbolism and 
terminology , This will lead not only to an enhanced lucidity 
in communication among professional workers but will also pro- 
vide for an accelerated development of chemical education 
itself. 

Before the specific problems of evaluation of chemistry 
curricula can be considered it is necessary to place chemistry 
in perspective within an educational system, to reflect on the 
aims and objectives of education as a whole and the part that 
chemistry as a subjeat can play in achieving such aims. Thus 
the report that follows is concerned not only with the design, 
construction and administration of chemistry examinations and 
evaluation of chemistry curricula but with the whole inter- 
related problem of curriculum change within an educational 
system* 
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I THE MAIN REPORT 



1 4 Aims; and Objectives of Education through Chemistry 

The aims and objectives of an educational system may he 
regarded as part of a cyclic process of curriculum development 
as indicated in Figure 1. 

Objectives may be seen and understood differently by pupils, 
teachers , curriculum designers and policy makers. Since this 
report is concerned essentially with evaluation, objectives will 
be considered mainly from the point of view of those involved in 
curriculum assessment. In this context it is possible to out- 
line broad olassif ications of objectives and this may be prefer ^ 
able to listing detailed objectives. Such amplification in any 
ease mast relate to the pupils for whom the curriculum is inten- 
ded and adjusted to the conditions prevailing in a particular 
society. 

Objectives may be broadly classified in two divisions - 
content objectives and behavioural objectives. 

The content objectives will vary with time and place as 
the subject develops and grows. The design of chemistry 
courses will have to reflect this change and the content 
should include carefully selected themes from chemistry to 
enable pupils to acquire an ordered view of nature and 
materials . It should also give pupils an appreciation of 
chemistry in everyday life, indicate the results of the appli- 
cation of chemistry to technology, agriculture and medicine, 
and show the relationship of chemistry to other sciences and 
culture* These themes must be treated in such a way as to 
give some awareness of the role of chemistry in terms of 
individual and social needs and of the scientific approach to 
problems. .. 

The behavioural objectives relate to changes which chem- 
istry teaching seeks to achieve in pupils* skills, attitudes 
and knowledge. These changes will be brought about primarily 
through pupil participation in experimental work leading to the 
development of three aspects of learning. First, the ability 
to use simple apparatus to observe changes in materials in 
chemical reactions and to record such changes in a meaningful 
way. Secondly, the ability to recall, classify and interpret 
information and experiences, to devise appropriate schemes for 
solving a practical problem, to apply previous understanding 
to new situations and to report and comment on matters of 
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chemical interest. Thirdly, to develop an interest and involve- 
ment in the subject and to gain satisfaction from it, leading to 
an appreciation of its value. 

Both the content and behavioural objectives of teaching 
chemistry need to be precisely stated in operational terms and 
will need to be modified accordingly when chemistry is consid- 
ered as part of general science as distinct from a separate 
subject. They must, however, seek to establish securely in 
students the desire and. capacity to learn throughout life. 



Interpretation in Ceylon 

'The following examples illustrate the way in which the 
objectives outlined above have bean interpreted in the educa- 
tional system, in Ceylon and conveyed to teachers, examiners and 
pupils . 

a) Objectives for Teach ere and Examiners 

In this aimlysis objectives are classified using- a modi- 
fied for® of H The Taxonomy of Educational Objectives" designed 
by Bloom et &1. (see bibliography) * 

A Table of Specifications (see Part II - Supporting Evid- 
ence, 1) listing the various items of the chemistry syllabus 
and analysing the questions set on each item :* s used to show how 
the following different objectives are tested in the examination* 

(i) Rftcs-ll s involves test items requiring a response util-^ 

ising primarily the recall of specific parts, terminology , 

* conversations , trends, classification criteria, method- 

ology, principles, generalisations, theories, etc. 

(ii) Com prehension! test items in this category demand a 

response requiring more than simple recall* An element 
of under s tanding is required and this may be of the 
following types? 

- translation of mathematical verbal material into 
symbolic statements and vice versa, 

- interpretation of data, statements (including simple 
explanations, summaries, etc*), 

- extrapolation from data, statements (i.e« extension 
of trends beyond the given data, determination of 
effects, corollaries, etc*)* 

(iii) Application s includes responses involving the application 
of abstractions in specific cases. The abstractions may 
be in the form of general ideas, rules of procedure, 
general methods, techniques, theories, eiq# 

A subdivision of this objective (iii) involves oaloul at i on . 
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( iv ) Analysis , ijnthesis End evQ.lua.tion * includes items 
testing recognition of explicit and implicit rela- 
tionships, structure; recognition of unstated 
assumptions; discriminations of facts from hypo- 
theses; organisation of ideas, statements, data; 
design of simple experiments; recognition of 
logical accuracy* consistency; comparison of 
theories and generalisations* 

b) Instructions to Schools: Q-levei Chemistry Examinations 

In the introductory notes to the new O-level chemistry 
syllabus (see bibliography) candidates are informed that there 
will be two papers — one of three quarters of an hour duration 
oonsi sting of short answer questions carrying 40% of the marks, 
the other of two hour duration. They are also advised that 
in their answers they will be expected to give evidence ofs 

(i) An unders tending of the facts, principles and theories of 
elementary chemistry, 

(ii) An appreciation and an unders tending of the relevance of 
elementary chemical facts, principles and theories to 
everyday life. 

(iii) A simple appreciation and an unders tanding of the methods 
of experimental science as applied, in elementary chemistry 

(iv) A familiarity with simple apparatus and 'experimental work# 
(The possibility of using simple materials, other than 
standard laboratory apparatus, to perform experiments, 
should be recognized. Teachers are advised to use such 
materials wherever they are relevant.) 



2. The Function of Examinations in Chemistry 

in an Educational System 

As outlined in the previous chapter the process of curri- 
culum development nay be regarded in a cyclic manner (Fig. 1). 
Having formulated aims and objectives, appropriate action is, 
required to achieve these goals. In turn assessment is used to 
determine to what extent and how successfully the aims and 
objectives have been achieved and to adjust both the objectives 
and action if necessary. 

Thus examinations as one means of assessment play a 
powerful and somewhat complex role in the development of an 
educational system. In various systems examinations in chem- 
istry are currently being used for some or all of the following 
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purposes, Iz is unlikely, however, that any one examination 
would be valid for all the purposes listed. 

The first three purposes listed are directed, at the pupils | 
the reminder are aimed at the educational system as a whole* 

In a well established system the main purposes of examinations 
may be simply the first three, but in a developing - system the 
other purposes can be of great importance in providing data on 
which judgments and adjustments can be made. In particular 
(vi) and {viii) can have a major formative influence on teach- 
ing and learning* . 

(i) To tost achievement of pupils* 

(ii) To indicate potential of pupils, 

(iii) As a selection instrument for higher education or 
employment, 

(iv) To assess effectiveness of teaching, 

(v) To diagnose pupils 1 difficulties, 

(vi) To stimulate and modify teaching and learning methods 
and attitudes, 

(vii) To modify future examinations, 

(viii) To spell out the objectives and content of the curriculum 
in operational and specific terms. * 

> (ix) To aot aa a sensitive and rapid steering system to 

adjust decision making at the aims and objectives level. 

(x) To reflect quantitatively the characteristics of an, 
educational environment . 

(xi) To locate constraints in. an educational system. 
Examinations in Ceylon 

* Chemistry examination in the Oeyloh educations*! system 
are used for all the purposes listed above in the following 'tfay* 

The Ceylon G-C.E. i 0 t and l A t level examinations are used 
as achievement tests and also for selection purposes ( (i) and 
(iii) ), In the absence of sui table I. ft. or aptitude tests 
the G . C . E , examination in science, mathematics and first lang« 
uage is also used for predicting the potential of pupils ( (ii) ) 
using correlations from the G.G.B. National Stratified Handosi 
Sample Data (see Part III -■ -Background Papers 2 and 3). 
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Th e e T fa c 1 1 v e - '£ ~ .v V : o a o \ . i r.. ’ i v ( i ™ ' } par t icular areas 
of the syii i peri'G i ci in general are meas- 
ured by Difficulty Indicia and Means from the G, O.E. 1 0* 

National 6-uraxXfioa Ranaoia D&xa, Likewise specif i o points 
in the _ ta&ohirg nzl.tTia a re iuM for diagnostic purposes 
C tv) } - to inn/. cat. o pr.pils ! clif ficultic :- and to suggest 
s*sinfoi‘ c fi.viOn t a n o. new tsacl'ii.ng oequenoes^ In-* service tr&in- 

ing courses sro aimed at specific areas of the syllabus as a 
corrective to weaknesses brought to light in the data. 

During the period 1 J6l~t>5 changes in the examinations in 
chemistry were daoignsd to stimulate changes in the teaching 
( (vi) ) n A greater coverage of the old syllabus was made 
possible by changing the e *. \w 7 aauor*. structures and also objec- 
tives sue. a as application, sia end synthesis (see Part III 

Background Paper 2) ware introduced* This period of change 
made it possible to introduce a major curriculum and examination 
reform in 19&5 * further changes in the design and moderation 
of chemistry examination. 3 ( { v 1 i ) } are brought about by refer- 
ence to the National St re., tiffed Hansom Sample. The teachers 
t*nd pupils may ref er to a prototype paper and marking schemes 
attached to the new syllabus end teaching schemes in order to 
become completely familiar v,ith tLs objectives and content of 
the curriculum in specific tenaa ( ( vili ) )* 

The way in which an examination can be used as a sensitive 
and rapid steering instrument ( (1 a) ) to assist policy making 
may be exemplified by the problem facing Ceylon in the mid- 
196 ®$ in the wish to extend science teaching facilities in 
rural u.tbb,b but with a sever© Xisvit on the' nurttr of university 
science graduate teachers) available* Ordinary trained (non- 
graduate) teachers were available and some were teaching at 
G.C.B* f U f level but doubt on their suitability for such work 
was widely expressed. Bata from the Rational Stratified Random 
Sample for December 1955 and 3,966 were studied and it was shown 
that these doubts ware* not .'reflected in the achievement of 
pupils,. Thus, in this specific context, the continued deploy- 
ment of non-graduate teachers at this level was regarded as 
permissible as an emergency measure. 

Bata from the (1*0.13. 5 €H examinations 1952-195B were used . 

to demonstrate the quantitative relationship of performance with 
educational environment and the isore recent data fro® the 
National Random Stratified Sr.njple are being used to measure 
constraints on the cdqc&tiorL&l system such as urban/ rural 
differences, sociQ*ecoromlo end educational status of parents, 
a© well as teacher qualification and experience ( (x) and(xi) ). 
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3 . A dmi n i s t r a t i v g an d Tea ohLng_ A o 1 1 on If e c e s sarj 

to .Achieve Ob j ec tives 

A « PL a se 1 - Implementati am DeBigner/Teacher 

Although the same person may be functioning in different 
roles as curriculum designer, examiner and teacher it is easier 
to separate these roles for convenience and conceptual purposes. 

There may be several intermediary links between curriculum 
designer and teacher, depending on the particular situation, 
type and eiza of the educational system but, direct or indirect, 
interactions between designer and teacher are moat important* 

Having stated the aims and objectives of the chemistry 
programme the implementation will depend on three principal 
factors - demographic, economic and social* Demographic fac- 
tors will include, among others, numbers of schools and 
teachers, their rates of increase and pattern of distribution. 
Economic factors will include the national financial commits 
ment to education, priority needs of scientific rnanpower at all 
levels, availability of additional resources from outside the 
country and supply of science equipment. Social factors will 
include societal attitudes to chemistry and its applications , 
readiness of the teaching profession for change, and patterns 
of authority and leadership at diff ©rent ‘ levels . 

The phasing and rate of implementation of change will he 
dependant on these factors. But the successful implementation 
of any programme will also depend on the effectiveness of 
obrnmuni cation between designer and teacher which will need to be 
clear, unambiguous and appropriate to the level and background 
of those concerned* Channels of communication should be 
established, if not already in existence, and developed to com- 
prise a set of clearly specified operations. Since no commu- 
nication system will function with complete eff icienoy it will 
be necessary to provice assistance and guidance in interpreta- 
tion for teachers. 

Concurrent with the establishment of effective communica- 
tion between designer and teacher there should be administrative 
action to allocate resources for implementation. Among these 
will be additional teaching staff, in-service training facilities, 
guide lines for teachers, textual materials, specifications of 
examinations and above all, time. Only when this management 
action has been taken can the second phase of implementation - 
teacher/ pupil change - be initiated. 
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B, Phase 2 - Inclement lx 

TBs teacher becomes aware of the alma, objectives sad 
content of a new chemistry prograram© through syllaousee, 
teachers* guides and schoneu of work, pre- ahtt ** serv 
training courses, examination papers and work themes, and 
textual or other resource material. It is assigned - ■ 

teacher is reasonably coin potent in the subject matter ^ d ± ffi1 »ie- 
the normal way ha would prepare lesson plan* designed to 
aant the ohjeotiv® a^d content oi vhe ne^ course. - llt±e 
assumed that ha bos appropriate working spaoe and facil 
for teaching chemistry.- 

In printing his material the teacher uses oral communica- 
tion, textual mateiialo, demonstration* , visual aids, pupil 

investigation, - discusaiour, class and homo oacer ^®*!* w^i.its 
curricula activities (such ao science hlubs, projeot. . and visits 

to outside laboratories' cud factories) end ? 3 ®* i JJ t ^ a * dittary 
Examinations comprise quo jtioh/ answer techniqV _ji 

lesson, periodic” tests end any final examination. ®h.s* *11 

provide learning situations and me y be used to diagnose pupi 
difficulties and to modify teaching methods. Moreoverthey ay 
be used to aps03S how effectively the objectives have been 

achieved. 

Im plementation i n ,0,o?.l<m 

IaplcBontetion of a*. ohraistry S'f^tS.h.r. 

B^eteta ha & he ? m effso‘o®d bj detailed gcheia^s , rare 

These not only listed the usual items in the syllabus _«a^ 
teaching outcomes for each section, s ^sted teaching procedures 

experiments. end activities ior pupxls an _ a s_ a ® appropriate 
lens of motivation, both;- for pupil and teacher, at appro pr_ 

points., Another inter sating aspect, is cha ,5°“® ? Jh^uroviaion 
which pupils are encouraged to undertake - thro g ' not^onlX to 
of simple apparatus, and chemicals whore necessary - not **ly . to 

faaiXiarise them „ith in ««jd* J^' t ^Jsf.r of 

chemistry plays a part, bu«» al^o to aseio-B ai 
classroom learning to everyday life. 

Assistance end guidance to teacher* are 
weekend study circles and vacation courses. _^ ls0 __ ,taff in 

such assistance is provided by t a staff of the Ministry of 
.consultation with the 'curriculum design staff 01 vne 

Education. 

Further detail, of the edainietrettr. end t«*»hlng.otlon 
carried out in Ceylon to achieve objective* ?£ ° h ® 7 

prS^me are given in Part II - Supporting -Evidence. 
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4 * Planning an Examination Scheme 



A* Introduction 

Before an examination scheme is planned it is essential 
to know the purpose of the ©xeminati on * / Various way© in which 
examinations in chemistry aro being used are given in Chapter 2 
but each co up try will have to decide within its own educational 
system which of these purpose© are intended and this informa- 
tion must bo submitted to the examiners* 

It is ale© important to have information on the character- 
istics of the student population far which the examination is 
intended « Thes© characteristics will include age groups of 

students f length of schooling and other relevant socio-economic 
and educational factors* They will also inolude the pattern 
of previous assessments based on earlier examination© and 
evaluation procedures* 

B # Information racuirod by the examiner 

Effective examining always used to be and often still is 
the inspired and intuitive work of successful teachers# How- 
ever t in recent years , concepts and techniques have Merged 
which permit professional educationists to work on curriculum 
design and evaluation* exposing some of the weaknesses of 
earlier methods of examining# This ha© enabled the diverse 
experience© of professional curriculum designer© and teacher 
examiners to reinforce each other leading to new concepts of 
examination construction and assessment • 

In order to design an examination it is essential for the 
examiner to have some basin inf ormwtion from the instructional 
staff or from those who have designed the curriculum. In 
addition to tho broad purposes of the examination and the 
characteristic© of the student population ^mcntiopad in Seotion 
A above) the examiner must know th© content of the curriculum 
and the pupil activities involved. 

One way of presenting courst content is by means of a 
written syllabus organised so a© to indicate major. topics and* 
sub-topics# There should be some clear indication in the 
syllabus on the importance and emphasis, as well as the time 
that has been devoted to the major segments of this course* 

The content specif loation should indicate the basic material 
required of all pupils and the alternate or optional material in 
the course * 

In addition to specifications of content. It will be 
necessary to indicate the levels of attainment expected of the 
pupils. In particular,* it will bn, necessary to specify in 
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raspect of each area of the content the level of behavioural 
aoLi eves ant ©rpected ( cognitive, psyoho-aotor and affeotive 
factors Bra ih©' usually accepted technical terse in. this 
context (sse glossary), 

Forthcraoi’e, relevant and pertinent information oan be 
givon by describing the teaching procedure and activities to 
o-hicli the pupils have been exposed to achieve the objectives. 

It is a growing pr notice in the planning of an examina- 
tion scheme to relate the content objectives, behavioural 
objectives and teaching activities, in a variety of ways. 

Tho xost esasoa way by which this specification is achieved 
i c by saeans of ‘ c, tvro— diBonsional grid, examples of which are 
given in Purt II — ■ Supporting Evidence, 1, 5 and 4» 

C. Available forms of aoeesement 

There nrs three broad categories of assessment* 

(i) Intornal aasesesont by teachers, whioh is an integral 
part of teaching. 

(ii) Assooeraent by an external body. This is essentially a 
tts*?aiiu\i pro cos & to assess attainment at the end of a 
couraa, 

(iii, ) Assasoaent in which a terminal assessment administered 
'by an external body is combined with an assessment made 
.by the teacher. * 

The teohniquea of assessment in all these categories may 
b© e. written exossination.', an oral test, a practical examination 
bs.B9d.ou written evidence or direct observation of pupils' 
activities. Various combinations of these techniques are 
possible. 

D, T he ap ^ gopgiate uae of various forms of examination 
'(see aiso ■pSrT^Il" - Background Papers, '6j 

(i) Written examinations differ widely .in form* One type of 
question is oallad fixed response , that is a question in 
which the candidate has to choose between some determined 

‘responses, •, o*g. *. 

When a weighed piece of magnesium is completely 
burned in air, which on© of th* following is 
obtained? 

(i) A black powder whose weight is greater 'than 
that of magnesium. 

(ii) A black powder whose weight is less than 
that of the magnesium. 
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(ill) A white powder whose weight is greater than 
that of the magnesium- 

(iv) A white powder whose weight is less than 
that of the magnesium- 

A second type of question is the single open-onded ques- 
tion to which candidates have to give a short answer 
having been determined by the examiner and included in 
the mark scheme, e.g.s 

Name two substances in the Ceylon home that are 
used as fuels# Give one instance where a liquid 
fuel is used as a solvent in the home* State 
whether the solute you have mentioned in your 
example has undergone a physical or a chemical 
change . 

Give one reason to support your conclusion, 

A third type may be called structured questions , In these 
the examiner has predetermined in detail the structure of 
the answers he expect© and the question ±b divided into 
parts that correspond to these answers. In other words 
a single complex situation' is broken down into a set of 
open-ended questions all requiring short answers, e.g»s 

In an experiment JO ml of chloroform is shaken with 
30 ml of NEj/HgQ of strengths (a) 1M. (b) *9M. 

('o) *Sm (d) .7M. (©) .6M„ in a conical flask 

as shown in the diagram. 

The mixtures were allowed to stand for 5 minutes 
and analysed to find out the amounts of NE„ in the 
two layers* * 




(1) What liquid is in layer A? 
What liquid is in layer B? 
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(2) How would you obtain 10 ml of layer B 
without using a pipette? You should 
drew a diagram if it will make your 
answer clearer, 

(3) How would you show that layer B contained 
ammonia? 

(4) What might happen if a solution of 
ammonia in chloroform was shaken with 
pure water? 

(5) What might happen if a solution of 
ammonia in water was shaken with a 
solution of ammonia in chloroform? 



Mark Scheme 

(1) A - water (1) 

B - chloroform ( 1 ) 

(2) Use burette (2) 

Technique (3) , . . , 

(3) Any moist (X) indicator (1) showing colour 

change (l) . 

( 4 ) Movement of ammonia from chloroform into 
water ( 2 ) 

(5) Movement of ammonia from chloroform to 

water (l) , , , 

Water — > chloroform tip 

Possibility of equilibrium (‘2) 

Total - 15 

A fourth category of question is commonly called eB.s ajr 
question. These questions usually require a fairly 
long answer, not necessarily in. the form of a prose 
essay sine© other forms of communication - mathematics, 
diagrams or notes - may be more appropriate. _ _ _ s 
category may be subdivided according to the ®*thod of 
marking. If the answers are marked by a ® . 

tured mark scheme largely predetermined by the examiner 
the amount of freedom of response ie less than if the 
answer is marked by impression with ® “ Q f® k 

scheme when the candidates have opportunities of sub- 
mitting more divergent answers. E,g,* 

A mixture of .hydrogen and nitrogen, left in a jar 
for several days, at room temperature, shows no 
appreciable amount of ammonia. What special 
reaction conditions are used in the manufacture of 
ammonia from hydrogen and nitrogen, to make the 
production economical? Sive one advantage of 
making the mixture of gases flow through the 
reaction chamber, rather than remain static. 
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It is obvicuc that if a.; srisFiee a candidate to 

answer iii a prca-otermi-naf way, fixed response and 

structured questions a ■: a the most useful, If, however, 
the examiner wishes to allow the candidate more freedom 
in answering, the free response questions are more 
appropriate , 

It should noted fixed response tests lend them- 

selves it. ere readily to fine specification of most of 
the objectives, to pretesting 1 and data processing 
techniques, particularly .hat of question analysis, 
and the quicker publication of results. 

They allow a wide coverage of both content and objec- 
tives and iho marking of such tests is highly reliable 
and requires little skill. Examining bodies, however, 
should think carefully about the possible effects of 
adopting this form of assessment from motives of 
administrative* convenience alone, because there are 
many educational qualities that fixed response tests 
are unlikely to measure. 

The "Easay * 1 type questions!, which are marked from a 
detailed marking scheme, require a special comment. 
While apparently allowing free response to the candi- 
date, in practice the answer, if it is to be success- 
ful, must fit a detailed response largely predetermined 
by the examiner. It may seem that if an examiner has 
already structured the question in his own mind it 
would be better to make this structure explicit to the 
candidates in the. question rather than to ask the can- 
didates to guess at the structure# 



% li ) Oral tests 



In some countries much usa is made of oral techniques as 
part of a total assessment of achievement . This has the 
advantage, not only of testing knowledge and understanding, 
but also of allowing direct feedback from examiner to 
candidate during th© examination, so that it closely 
resembles a teaching-learning situation, particularly in 
chemistry, if it is associated with a practical task# 

|Mi) Assessment of practical work 

The assessment of skills, abilities and attitudes in 
practical work requires special mention# The marking of 
written evidence after a practical exerci s© . has , obvious 
defects as & tae&ns of assessing the practical work of the 
candidate# Examiners may wish to consider the use of a 
direct observation technique for this purpose in the form 
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of a continuous assessment of practical work by teachers* 

It should be pointed out that this assessment involves 
all three domains t cognitive and affective as well as 
the psychomo tor behaviour. 

There is some evidence to support the view that a pupil 
uses abilities when he is carrying out experimental work 
which are* to some extent, distinct from those used in 
non-practical situations* (A suggested specification of 
these objectives is given in Part II - Supporting Evidence, 
4, together with some information on the measurement of 
practical ability by continuous assessment, and a brief 
discussion of the problem of moderating these assessments*) 

If, however, only the psycho-motor dimension is to be 
evaluated, there is much experience in industry where the 
evaluation of skills is extensive* 



O 
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E , The effect of different forms of assessment on the teaching/ 

learning situation 

The emphasis placed on the different techniques of assess- 
ment differs between internal and external assessment. Assess- 
ment within a school relies heavily on oral and other direct 
observation .techniques, while in external assessment the empha- 
sis is on written examinations and tests.. 

It is also apparent that in external examinations there is 
a general trend, at present towards a more widespread use of 
fixed response questions in objective tests (so called). This 
trend towards a more rigid candidate-response in examinations 
is in contrast to recent curriculum reform which gives greater 
freedom of response, and initiative to the pupils in their 
lessons, , Taken to' their extreme, these two trends are not 
compatible, and the dangers ih this situation can only.be 
countered by the use of more balanced forms of final assessment 
in which more than one technique of assessment is used* 

If examinations are to have the desired backwash effect on 
education , the techniques used in examining must resemble those 
used in teaching; so that if a teacher finds it necessary to 
use more than one form of questioning the examiner should do 
likewise. The use by an examiner of one technique alone may 
well lead to coaching in schools in this technique to the exclu- 
sion of other, and equally desirable, forms of questioning 
between teacher and pupil. 



One obvious example of the effect of a- form of assessment 
on the teaching/ learning situation is that of direct observation 
of the pupil over a period of time. If this is carried 'out by 
the teacher, and if the outcome is used as a part of the final 
assessment of the pupil, a change in the relationship between 
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pupil ana teacher is almost inevitable. A pupil who kno« 
that his teacher is also his examiner is oound to view his 
teacher in a different light. This change in ^eiationship 
may not be undesirable, but it should not be ignored by those 
wh o wish to introduce some form of continuous assessment which 
will subsequently be used by external authorities. 



Administr ation and Execution o f Examinations 
*" in.~~Chemi s try 



appropriate individuals to examination 



O 

:KLC 



A , Th e appointment of 
boards 

If the examination is truly to be an instrumen ^/educational 
cation, if it is to serve as a means of assessment oieducatio 
achievement and as a means of making aajustments . 
tional process, then the composition and 

examiners and examination boards must be carefully considered. 

An examiner or a moderation board member should 

only long experience in teaching and * strong moral 

iect field, but. more importantly, he should ieei & sv : f , 
obligation of being open to new ideas and to innovations in the 
educational procesf. He should consider the examination not 
only as an instrument of performance but as a means for 
tional reform# 

Steeping with this Belf-oritical spirit, several practical 
steps can be recommended to examination boards. - 

(i) They should establish liaison with members of the scien- 
( tific and technological communities that *5°" 

tl B3 ta to serve (for example, with doctors and health 
officers, with engineers, with research scientists, wit 
industrial isanagarSf etc*)* 

(ii) They should maintain working contact with university 
( } scientists and with science curriculum design groups 

where these exist* 

(ill) They should ocoasionally consult at, international expert 
on examination construction and use. 

( iv) They should keep abreast of the growing specialised 
U ; literature on assessment and examinations in science. 
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th **a Th l 8 ° n3 +[° T a '' reBSXn S these points and recommending 
hese steps are that ir soma countries - indeed in many oeun- 

tit ® s ~. , th ® Examining Boards have cut themselves off from such 
imulation. As a result they tend to become a positive 

f f in ®3 ation aQd a source of discouragement and 
frustration to those who are striving for curriculum develop- 
J Ms . c&n be particularly acute in the context of a 
rapidly developing scientific discipline such as chemistry. 

B * Sequence of operations 

Having designed a plan and an organization, the examiner 
is ready to construct and administer the examination. The 
general sequence of operations and the various forms which 
each operation may take are described below. 

. + ■3 he J — ■ flHQ n °q of operations may be common to all countries, 
but the form of each operation is likely to vary from country 

in examining e P en ding upon their aims, resources and experience 

li£p*r$_ Jz. is arL attempt to show the sequence and possible 
f™ ^ ° pe l ati ?f diagrammatie&lly « The sequence of operation 

is shown vertically and, for the sake of completeness, a con- 
struction of the specification is included, although this has 
aircady bean discussed in a previous section. 

The various for m-? of operations available to an examination 
organisation are shown horizontally. Those who are about to 
set up their own examinations for the first time will need to 
choose which form of operation is most appropriate to them,* for 
convenient usage the forms of operation are arranged from left 
to^nght in order of increasing difficulty. By their very 
nature, the order reflects, in part, the historical development 
of examination construction. It should be noted, however, that 
a country need not evolve its examination structure by passing 
progressively through the sequence shown in Figure 2. Rather 
it should choose that form of operation which is most appropriate 
to the present circumstances . 

illustrate this point the development of operations in 
chemistry examining in Ceylon over the past eight years is shown 
in Figure 3, The reader may find it helpful to draw a similar 
diagram for the examination system in which he is interested, 

C. Writing the Questions 

The first job of an examiner is to arrange for some ques- 
tions to be written and the simplest way is for one or two chief 
examiners to write them. For short answer and free response 
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FI GURE 2 

Sc g uenoo o £ Oper a t lo oja 

shoeing sequence (vertically) and form (her! £ on tally) of operations availab.lt 

to .an examination organisation 
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The devolopiaeBt. A n Pay -ton of operations o-\ tr .in.j'i?. In Fig ure 2 
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questions this method worn: quits although there is 

always a danger th.*.t tb® examination, will reflect personal 
likes and dislikes of the examiner. 

If it is proposed to use fixed response and structured 
questions, however, examining authorities may find that the 
use of one or two Question writers will be effective for only 
a short time. The construction of such questions requires 
skill t experience and, above all f plenty of time. And if 
feed-back from classroom to examination is to be attained, the 
use of one or more groups of Question writers, directly_in 
touch with teac hin g in. Boh ools becomes desirable. 

The traneition from the use of one or two question writers 
to a group of question writers is at present taking place in 
Ceylon, where the first step to bring about this change has 
been the training of question writers* This was the aim of 
an evaluation workshop conducted in Ceylon in August I960, a 
report of which is given in Fart III - Working Papers, 6* 

This workshop is not presented as a model, but rather as an 
example of one attempt at the sort of training which is 
necessary. 

The selection of question writers is am important part of 
the process of building up an examination* It should not be 
taken for granted that because a person is a good ohemist* or 
a good teacher, or both, that he will necessarily be a good 
writer of questions for a chemistry examination. It is only 
by judging efforts in the actual construe t ion ^of questions that 
good question writers can be found? and, if it is thought that 
examinations should reflect and encourage the desired object*, 
ives and approach to education in chemistry, then the more the 
writers are in tune with actual teaching situation® the better* 

/It cannot be deni ad that the writing of fixed response and 
structured questions is difficult and tim© consuming* and 
writers before long may run out of ideas. It may appear * 
therefore, that the expenditure of good questions year by year, 
never to be used again, is wasteful* For this reason it is a 
growing practice to collect a pool, or bank, of questions, 
properly classified, which can be used when required, and 
possibly used more than once. To extend this idea, there 
seems no reason why on international pool of questions* at 
various levels of chemistry should not be formed (it should be 
noted that pre-testing, if required, would have to be carried 
out again on an appropriate population). An international 
pool of questions could greatly aebelerats the growth of exam- 
ination construction in developing countries* It oust be 
pointed out, however, that the formation of a pool of qhestienB 
requires sort complex systems of filing and seourlty than does 
other forms of examining, and this should be borne in mind when 
considering the administrative 'requirements of a new examination 
system. 
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Q, Critical review of Questions 



The writing of questions is only the first part in the 
construction of an examination. The next is a critical 
review of the questions, a stage which is particularly import- 
ant for questions of the fixed response and structured type 
in which much chemistry is often contained, in the question 
itself. If question writers are inexperienced, there will 
be a large wastage of questions at the review stage, often as 
high as three questions out of four. 

The first critical review of questions is often undertaken 
by a small group of examiners, but the practice varies from 
country to country. A possible set of criteria for a first 
review has been worked out for use in Ceylon examinations in 
the Ceylon Workshop and is set out below. (The list is not 
interned to be comprehensive and other factors may have to be 
considered, particularly some technical factors in the construc- 
tion of fixed response questions. If necessary, the reader 
should refer to standard publications on test construction - 
see bibliography * ) 

(i) Accuracy of the chemistry. 

( ii ) Dimple errors of construction, e . g . language, 
ambiguity, obvious clues, etc* 

(Hi ) The appropriateness of the level of study. 

(iv) The time required to read the questions. 

(v) The specif i cation of the question* 

(vi) Its effect on the teaching/ learning situation. 

Construction of the first draft of examination paper 

The reviewing process (sometimes called shreddingj may be 
repeated by other similar groups of people. Then the examiner 
is in a position to construct the first draft of the examina- 
tion paper. 

In doing so, the major factors, in addition to those given 
above, which must be kept in mind ares- 

(i) The specifications. 

(ii) The range and distribution of difficulties of the 
questions . 

(iii) Duplication of questions. 

(iv) Overall time required to complete the paper. 

(v) The relevant weighting in terms of marks for the 
various kinds of questions and the weighting 
within each question. 

(vi) Alternative questions and questions from which a 
choice can be maae should be equivalent as far as 
objectives, time required and difficulty are 
concerned. 
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p. Moderation of papers 

Another revie* of options and of the text as a *hole 
eiffiilar to that oonduoted by the aramnar, .ay no. he under - 
?iken by another group, normally called mod«rat or|, before 
the publication of the examination in ite final form. 

For thie meeting of the moderators the following materials 
would be required*. 

(,i) The draft examination. 

The table of specifications. 

Uii) The marking schemes. 

fit is assumed that the background material available to 
the examiner is also available to the moderators.) 

Pre-testing of questions 

The sections D, E, E above are based on the 
that pre-testing of questions is not feasible. However, 

“e educational ey.?e»a it i. poaaibl.toc.rryoutpre- 
testing of questions, particularly of the fixed response yp 

Pre-testing is essentially a trial of questions on a 

representative fample of pupils, the ob ^*Hhe*foL o/ 
vide the examiner with an additional tool in the of 

quantitative data, which will assist him' to refine his 

examination. 

For details of pre-testing, readers should refer to the , 
i d but they may find of interest the procedure 

Ml Srit tLe In Ceylon in 1968 aa a preparation 
oJt^a Evaluation Workehop hold in August of that year (ea. 
Part III - Background Papers, 6). 

Among the main data concerning an individual fixed res- 
ponse question which arise in pre-testing are* 

(*\ Percentage response to each sub-section (distrao- 
. tor) and a difficulty index (D.I.) of the correct 

answer, 

(ii) The disori mination coefficient (D.C.), 

mu* difficulty index tells the examiner the peroentage^of 

u _i is w ho responded t o^a question in the pre-test in the way 
pupils woo r P . . difficulty index means few 

the examiner expected. A low aiiiiouxvy 

questions answered correctly. 

The ooeffloient ‘« 11 . ^. .raainar the_ 

correlation betweon tha performance J ? h . 

ticular question with the performance of candidates 
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test as a whoi*. Neither the difficulty index nor the dis- 
crimination coefficient tells the examiner that the response 
was educationally desirable nor does it tell him that the 
chemistry in question was sound. Thus, it will he found in 
the examples given below that a pre-tested question whose 
difficulty index and the discrimination coefficient were quite 
satisfactory had to be rejected on other grounds. 

With the aid of this additional data moderators would then 
proceed in the usual way. (It is assumed of course that both 
examiners and moderators are competent in interpreting pre-test 
data, ) 

H , Critical review of some fixed response Questions from the 
Evaluation Seminar h olc T in Ceylon in 19o8 

Questions 

I, Sulphur powder and iron powder were strongly heated 
together. When tested for iron and sulphur, their 
properties were WOT evident. This is because i 

(i) a mixture was. formed. 

(ii) a compound vras formed, 

(ill) an element was formed. 

(iv) a physical change had occurred. 

2 . An element reacted with cold water displacing hydrogen. 
This element is most likely to be: 

(i) high up in the activity series. 

(ii) just above hydrogen in the activity series. 

( iii ) just below hydrogen in the activity series, 

(iv) vary far down in the activity series, 

: 

3. .Whenever a liquid becomes a solid which one of the 
following can he stated as being certain to occur? 

(i) Arrangement of particles in & regular pattern - 
(ii) backing of particles into a smaller volume. 

(iii) Absorption of thermal energy from the surroundings, 

(iv) Decrease in. the kinetic energy of the particles* 

-4, A gas showed the following properties. It immediately 
extinguished a glowing splint. It had no reaction with 
Ximewater* When metallic copper was heated in the g&s* 
no visible change was observed. Which one of the 
following could the gas be? 

(i) Sulphur Dioxide?. 

(ii) Nitrogen. 

(iii) Carbon Dioxide. 

(iv) O^rgen. 
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NaOII + 


H 2 S0 4 


= KaHSG 4 + 


H 2° 


+ X 


cal . 




HaHSO^ 


+ NaOH 


« NagSG^ + 


H 2° 


+ Y 


cal . 




2 NaOH 


+ h 2 so 4 


- Ka 2 S0 4 + 


2H 2° 


+ 2 


oal • 



The above equations indicate 2 methods by which sodium 
sulphate may be prepared. Which one of the following 
describes the thermal energy relations of the two pro- 



cesses ? 
(i) X 




Z 


- Y 


(ii) Y 


+ 


z 


» X 


(iii) Y 




X 


- z 


(iv) X 




Y 


«• z 



6, GaseB A and B have the following physical properties. 





Solubility in water 


Boiling 

point 


Density 
: (relative 
to air) 


Gas A 
Gao B 


70 g per litre 
15 g per litre 


-110°C 

2 ? C 


0.8 

1*2 

[ „ — 



In order to collect as pure a sample of B from a mixture 
of A and B with as little wastage of it as possible, 
which one of the following collection procedures would 

you. adopt? 



A & B 
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What weight of potassium chloride (molecular weight 74*5) 
would be formed when excess hydrochloric acid reacts with 
17 g of potassium nitrite KNO„ (molecular weight 85, the 
salt of the weak acid ni trous acid) ? 

(i) 7.45 S 

(ii) 8.5 g 
(Hi) 14.9 g 
(iv) 17 g 

In which one of the following solutions will chlorine NOT 
be liberated? - • 

(i) Gold concentrated hydrochloric a old and hot concen* 
t rated sulphuric acid* 

(ii) Cold concentrated hydrochloric acid and bleaching 
powder* 

(iii) Gold concentrated hydrochloric acid and crystals 
of potassium permanganate* 

(iv) Concentrated hydrochloric acid heated with 
manganese dioxide. 

The diagram represents an electrical circuit involving an 
oxidation reduction cell C, a voltmeter V, an ammeter A 
and a resistance R* 
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Carton elec trod <5 
Porous pot 



KMnQ, solution 

4 




solution 



In which of the following ways would variation of 
concentration of the permanganate solution be 
expected to affect the voltmeter and ammeter 
readings? 

(i) The KUirmeter reading will change but the 
voltmeter reading will remain un changed, 

(ii) The voltmeter reading will change but the 
ammeter reading will remain unchanged* 

(iii) Both the ammeter and the voltmeter readings 
will change* 

(iv) Neither the voltmeter nor the ammeter readings 

will change. 



10, When four substances A t B, G , D were heated in air 
the following observations were made* Which one 
of the following observation would alone suggest 
that the substance concerned was NOT an element? 

(i) A liquified but no new substance was formed, 

(ii) B burned producing sulpher dioxide, 

(iii) 0 burned producing only an oxide. 

(iv) D neither liquified nor prodxxced a new 
substance. 



O 
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Review 



Item 

No. 


Content 

code 


Objective 


Accuracy 


Errors 


Level 


Time 


D.I* 


D.C. 


i 


4 


(1) 




- 






.69 


.54 


2 


14 


(i) 


v/ 


Needs 

editing 






• 71 


.50 


3 


2 


(2) 










- 


- 


4 


29 


(2) 




Stem may be 
shortened 


x/ 


x/ 


» 65 


.39 


5 


27 


(3) 




Cons true ti - 
on (clarify) 


n/ 




.44 


.39 


6 


2 


(4) 




Obvious 

clues 

U) * (ii) 




x/ 


.46 


.44 


7 


18 


(3)’ 


v/ 


Use formula 
weight in- 
stead of 
molecular 
weight* 
Irrelevant 
data. 


v/ 


n/ 




! 


8 


' 29 


Changed 
from (2) 
to (1) 


Quest- 

ionable 


Obvious 

clues 


vx" 


x/ 


.53 


.40 

\ 

b 


9 


If 


(3) 


v/ 


R omitted 
in diagram 


Too 

diffi- 

cult 


Too 

long 


.46 


.28 ■ 


10 


4 


(4) 


v/ 


Needs 

editing 






• 34 


.44 



Multiply by 100 to obtain percentage figures for difficulty index 
Content Cedes 



TABLE OF SPECIFICATIONS 



CONTENT COPE 



O 

ERIC 



Nature and scope of chemistry. 
Substances' and their properties- 

33 



1 

2 
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3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 
21 
22 

23 

24 

25 

26 
27 
26 
29 



Solutions* 

Elements, mixtures and compounds* 
Physical and chemical change* 

Burning. 

Air* 

Oxygen 

0 on d i t i. on s for c oxnbus ti on . 

Oxidation ~ reduction 1« 

Action of acids on metals. 

Hydrogen „ 

Lawsr of chemical combinations. 

Action of water on metals* 

Atomic theory. 

Classification of reactions* 

Chemical calculations . 

Action of acids on salts* 

Electrolysis * 

Ionization and equilibrium. 

Corrosion * 

Atomic structure. 

Garb an ch em i s try . 

Chemicals from air. 

Chemicals from 3ea* 

Chemicals from soil. 

Energy changes in chemical reactions* 
Radiochemistry . 

Oxidation - reduction II. 



Objectives code : 

(l) Recall. 

(2; Comprehension. 

(3) Application* 

(3) f Calculation. 

(4) Analysis, synthesis and evaluation* 



Additional comments on the learning/teaching situation 



Q. 1 . 



Q. 2. 




Question based on actual experimental work done in 
class. Encourages pupil activities/ teacher demon- 
strations, making observations and drawing conclu- 
sions from experimental data. 

May be answered without actually doing the experiment, 
but it encourages the teacher to introduce simple 
experimental work. The reference cold water suggests 
a further investigation of the effect of variables 
such as temperature. The item could be made more 
concise as follows: - 

An element The location of the element 

in the activity series is most likely *to be: 
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(i) high above hydrogen, 

(i±) Just above hydrogen. 

(iii) just below hydrogen. 

(iv) far below hydrogen. 

Brings up the idea of using a model in chemistry to 
explain the properties of matter. When generalisa- 
tions are made, as suggested in this item, the 
teacher should be encouraged to be conscious of any 
anomalous situations such as the behaviour of water 
in this instance . 

The item could be more concisely stated. Encour- 
ages pupil activities/teacher demonstrations, make 
observations. 

Needs minor editing, The question indicates two 
methods, but gives three equations. It is 
necessary to indicate that the first two equations 
refer to one process. 

This question draws attention to broad generalisa- 
tions and applications . 

Emphasises the need to confront pupils with the 
ability to cope with new situations. Encourages 
training in suitable designing of experimental 
set-up to solve problems and apply general prin- 
ciples in ©hemistry to new situations. Unfor- 
tunately, choices (i) and (ii) are not good dis- 
traetdrs. 

Editing* 

(i) Delete - "the salt of the weak acid, nitrous 
acid" (irrelevant data). 

Use formula weight instead of molecular 
weight since the substance is ionic. 

Use 8.5 instead of 17. It will be 
easier for the pupil to recognize the 
relation between the formula weight of 
KNOg and the given weight. 

Item encourages the teaching of chemical 
calculations. 

Example of an unsatisfactory question. (ii), 

(iii) npd (iv) are obvious clues, actually carried 
out by pupils, but (i) is not done and on that 
basis alone it- may be selected. Accuracy of the 



Ui) 

(iii) 



FILMED FROM BEST AVAILABLE COPY 



- 50 - 



chemistry is also questionable . Any attempt to 
test (i) experimentally by a pupil may be 
dangerous . 

Q, 9 , Question rejected.. May be too difficult, The 
examiner probably wants to test the understanding 
of variation of voltage with, change in concentra- 
tion, which may be too difficult at this level. 

In addition, he has coupled to it a problem 
involving variation of both voltage and eurrent 
which makes it even more difficult. 

The diagram does not show 'h*. 

Q.10, baitings Insert the word 'separately* after 

•heated*. A good question which cells for analy- 
sis synthesis and evaluation of experimental data 
and making conclusions. 

I , Rejection of fixed response questions 

Rejection of questions may be effected at a relatively 
unsophisticated level on the following grounds which are illus 
trated by examples, 

(i) Aocuraoy of the chemistry 

Copper reacts with pOJ& nitric acid according to 
the equation* 

JCu + SHKOj - 3Cu (N0j) 2 +.4H 2 0 + 2N0 

The volume of nitric oxide at STF which ia formed 
from 6J.5 g of copper would bet- 

(i) 2a, 4 litres. 

(ii) 44*8 litres. 

(iii) 14,9 litres. 

(iv) 7.5 litres. 

Comment 

This reaction i© aor© complicated than indicated 
by the equation given in this item, 

(iij Simple errors smofa as language, ambiguity, obvious 
clues, ©to . 

The solubility of potassium chloride at various 
temperatures is represented below*- 
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Which one of the following points indicate a 
saturated solution? 

( i ) B and B only . 

(ii) B f 0, B and E, 

(iii ) C and B only, 

(iv) A only. 

Comment 

Both diagram and the stem of the question are 
confusing. The correct response does not 
appear as one of the choice. • 

■ (iii) Appropriateness of the level of study 



i 


ii 


iii 


iv 


V 


vi 


vii 


viii 


0 


a . 


b 


G 


d 


e 










f 


e 


h 


i 


3 










k 


i 


m 


n 


o 











Members of the first elements in the periodic 
table are represented above. 

Which one of the following pairs of elements is 
LEAST likely to show resemblance in their cherai- 
cal properties? 

(i) f, k, 

(ii) g> 1» 

(iii) f, 1. 

(±v) k, g. 

Comment 

In the Ceylon teaching manual this depth of study 
is not expected to. be reached. 
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(iv) Reading time of question s 



A pupil weighed 10 g sugar into each of two 100 ml 
beakers A and B. The sugar in beaker A was in 

the form of a fine powder* The sugar in B was in 

the form of a single lump# He simultaneously and 
carefully poured SO ml water into A and B, without 
stirring the contents * At intervals of 5 min. 
duration, five-drop samples of the liquid from 
each beaker were withdrawn and their separate 
sugar content determined. If these results are 
graphically represented, which one of the following 
would best depict the results of„. the experiment? 







(iii) (iv) 

Comment 

An average pupil may take as much as 1-^ minutes to 
read and understand the stem only of this question. 
In this paper 45 minutes are allocated for answer- 
ing 40 questions and the time taken for this ques- 
tion is out of proportion. Furthermore, there is 
an sir of unreality in the question; it seems 
unlikely that the writer of the question actually 
performed the experiment. 

(v) Effect on the teaohing/learning situation * 




When HgS is passed into aqueous solutions of the 
following groups of metals which group will produce 
only black precipitates? 

38 
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(i) 


Pb f 


Lu , 


Cd. 


(ii) 


Cu, 


Co , 


Sb. 


(iii) 


Pb, 


Si, 


Cu, 


(iv) 


Mi, 


Cu, 


Zn , 



Comment 

This kind of item encourages memorising information, 
which in the context of school chemistry can only 
b© regarded as trivial. 

At p, more sophisticated level when pre-test statistical 
data are available further criteria may be- -used a© exemplified 
below, 

(1) Low difficulty index 

When a finely divided metal A is mixed with the oxide of 
metal B and heated, no change is seen to occur. When 
the oxide of metal B is heated, with a metal C, metal B 
and oxide of G are formed, Y/hen metal G ia added to a 
solution of a salt of A, particles of metal A are 
deposited. From these observations the increasing 
order of activity of the metals B, C and A ist- 

U) c, B, A, 

(ii) B, C, A. 

(iii) 0, AB, B# 

(iv) A, B, 0, 

Comment 

This item though discriminating positively (0.23) has a 
very low difficulty index of 0.13* This is probably 
due to the length of the item, the complexity of infor- 
mation and the use of symbols# 

(ii ) Low discrimination oe efficient 

What may HOT always ba expected in a balanced chemical 
equation describing a chemical reaction is that the sum * 
of the*- 

(i) weights of the products equals the ©urn of the 
weights of the reactants# 

(ii) volumes of the products equals the sum of the 
volumes of the reactants. 

(iii) number of atoms cf the products equals the sum of 
number of atoms .< £ the reactants# 

(iv) atomic weights o ’ the products equal© the sum of 
the atomic weigl’ bs of the reactants. 
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Coma-en t 

This 1 tem has discrimination coefficient as low as 
- 0.55 although the difficulty index is 35^. tter9 
the stem in the negative form may have been the 
significant contributory factor. 

(iii) Non functioning distraotors 

Three similar burning tapers were plunged simultane- 
ously into 5 separata cans containing coconut oil, 
petrol and kerosene respectively. Petrol caught fire 
at once, but HOT the other two. This was probably 
due to the faot that?- 

(i) the temperatures of the three tapers were not 
the same. 

(li) the ignition temperature of petrol is higher 
than that of the other two. 

(iii) the ignition temperature of petrol is lower 
than that of the other two, 

(iv) petrol has a lower density than either of the 
other two. 

Comment 

The first response does not appe ,r to function 
adequately as only 0.6fo of the test population 
selected this choice. The fact that the first 
choice is a wrong response may be self evident from 
the first few words of the stem. 

(iv) Over functioning distraotors 

Milton is used to remove ink stains. The agent res- 
ponsible for the bleaohing is active* - 

(i) chlorine. 

(ii) oxygen. 

(iii) hydrogen. 

(iv) hydrochloric acid. 

Comment 

72^ of the test population selected the first choice 
(an incorrect choice) whereas only 20$ responded to the 
correct choice.' This type of over functioning is 
sometimes due to the poor wording of the item, as borne 
out here. 

SPECIAL NOTE 

Although an item may appear acceptable on indices of Po- 
test data, it may still be possible that the item is_un.ui table 
because of errors which, escaped a first scrutiny. This could 
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happen because the target population ignored the errors and 
responded correctly. But still the question must be either 



re j ec ted 


or re-written 


even 


though the i 


tern appears to 


function 


well * 














e 


. g. NaOK + 


H 2 


S °4 


-•> KaHSQ, 
4 


+ HpG 


+ X 


cal . 




h'aHSO 

4 


4- 


h 2 so 4 


- Na_SO . 
2 4 


+ e £ o 


+ Y 


cal • 




2 NaOH 


4- 


K 2 S0 4 


- Na-SQ. 
2 4 


+ h 2 o 


+ Z 


cal . 



The above equations indicate two methods by which sodium 
sulphate may be prepared* Which one of the following des« 
cribes the thermal energy relations of the two processes? 

(i) X - Z « Y* 

(ii) Y + 2 - X. ' 

(iii) X + Y - Z. 

(iv) Y - X « 2. 

J, Administration of examinations 



Organisations for the administration of examinations are 
bound to vary greatly. For the purpose of this report only 
those aspects of administration which may directly affect 
decisions on the form of examination are considered. 

■A general pattern of examination administration may be 
classified as school based, regionally based, and nationally 
based. If the assessment includes some form of continuous 
assessment it will entail an extra administrative load on the 
schools . 

A particular difficulty in some countries may be the 
need to translate an examination into several languages. 

This produces problems not only in translation of idiom but 
also in translation of technical vocabulary • Other problems 
in question setting arise when there are different cultures 
within a country , particularly when referring to chemicals 
which may be found in one cultural background but not in 
another > 



Security 

Most examinations at present remain secure before admin- 
istration. There are exceptions, the most important of 
which ,t& some oral and practical examinations and continu- 
ous assessment of practical and other work which is the 
normal part of a chemistry course. 

In some secure examinations, pupils may be allowed fore- 
knowledge of a part of the examination or relevant materials 




b!. 
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of the examination | Tor example, open-book examinations and 
examinations in which books on data are allowed. At the 
other extreme some examinations, particularly some of the 
fixed response type, are kept secure before and after the 
examination, This raises further administrative difficul- 
ties . 

Printing and handling of examinations 

The degree of administrative difficulties in this area 
depends largely on the form of the examination and question 
used, For example, the use of fixed response and structured 
Questions, giving a lot of various types of- information in 
the question itself, inevitably leads to a bulky and more 
complex examination, The sheer bulk raises administrative 
problems in printing, packaging, storage, despatch and security. 

Marking 

Marking may be in three forme - from a fixed key, from a 
finely structured mark scheme and by impression, these being 
in the order of increasing difficulty. 

A special comment is required on impression marking. 

This is a grading process rather than a marking process and 
is normally conducted by more than one examiner with general, 
rather than detailed, suggestions for marking. Such marking 
may call for a close examination of standardization procedures. 
People qualified to mark by impression are not always' easy to 
find. A further administrative difficulty in impression 
marking arises from the need to post scripts from one marker 
tp another in a short period of time. 

Where the same question appears in more than one lang- 
uage, ‘in an examination paper and there is error in one trans- 
lation, the examiner must classify all translated questions as 
faulty, because bilingual students may have read faulty and/or 
the eorreot one. 

K. Analysis, interpretation and Reporting of results 

*,fter marking, the examiner has available for analysis the 
marks of each candidate in eaoh question, the total marks In 
each section, and the total mark in the examination as a whole. 
The common practice is to use the total marks scored as a 
measure of attainment. This implies that marks obtained on 
different sections of the. test which were designed to measure 
different objectives can bo added to give a meaningful result. 
The candidates are then put in order of me^lh according to the 
total marks. The simplest interpretation of this order of 
merit ie to classify candidates into those who have passed and 
those who have failed. . But interpretation may ‘be used for 

O 
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many different purposes (see Chapter 2 ) and one may ask if it 
is the propei' function of the examiner to decide who has 
passed or failed. 

A second and more refined form of analysis puts candi- 
dates into grades indicating more clearly a hierarchy of per- 
formance by the candidates in the examination as a whole , but 
gives no further information on the candidates 1 performance 
in the different objectives of chemical education, in other 
words a profile of the candidate. To do the latter would 
require a more elaborate but possibly more profitable form of 
analysis, interpretation and reporting of the results , in the 
light of the objectives cf chemistry teaching. 



6, Strategy and Tactics for Curriculum Reform 

The major assumption underlying recommendations and 
suggestions in this report is that curriculum development and 
examination reform are no longer one-stage operations* The 
pace and pattern of Chang© is such that insti tutional arrange- 
ments which have been dieorimlnately designed to cope. with 
change have become imperative* One feature which should 
characterise such institutional developments is the provision 
for enhanced participation by teachers of chemistry, as well 
as professional chemiata from universi ties* health, agricul- 
ture and industry. 

Just as in chemical research, team work is increasingly 
replacing individual isolated efforts, so in curriculum 
development and examinations reform the need to provide for 
co-ordinated group effort is becoming necessary. This is 
being achieved through the establishment of curriculum 
development centres in which participants from many fields 
pool their experience and insights in the preparation of 
curriculum materials - 

One of the prim© functions of such activity may b© the 
reduction of the T !ag time 1 between the ©mergence of new ideas 
and techniques , and their application in classroom situations. 
Another important function of curriculum reform groups is to 
sustain the momentum created during the process of renewing 
curricula, bo that curriculum development groups might under- 
take to keep their materials, practices, methods and tech- 
niques in a proooss of continual reappraisal and readjustment. 

A danger to be avoided is the elevation of the 'new 1 
material and practices to the status of a new dogma. # Hence, 
the need exists for some sensitive feed-back of information 
as to the direction in which the system is moving and a 
discriminating examination design provides such a monitoring 
and steering device. 
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Institutional organisation for curriculum reform and 
examination design will not in itself ensure the diffusion 
and adoption of the new ideas. There is a need for more 
understanding of the process by which change comes about in 
school chemistry teaching. Inevitable consequences of 
curriculum reform are that relationships between all parti- 
cipants in education will need to be modified and adapted to 
permit growth and development * 
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APPENDIX A 
GLOSSARY 


Achievement 

testi 


A test designed to measure the performance 
of a pupil in relation to the expected 
outcomes of instruction. 


Affective 
behaviour a 


A pupil f a emotional feeling tone# This 

may include internal feelings and emotions f 
or may be expressed in overt behaviour 
marked by an immediate impulse to act 4 


Attainment 

tests 


See Achievement test* 


Behavioural 
objectives i 


The analysis of the objectives of a oourse 
in terms of intended changes in the ways in 
which the pupil will feel and act. 


Cognition! 


An intellectual process whereby a pupil 
become b aware and obtains all the various 
inodes of knowing, including all the forms 
of reasoning. 


Content 

objectives* 


The listing of the major divisions and sub- 
divisions of the subject matter to be 
oovered in a oourse. These are often 


* 


included or implied in a syllabus. 


Diagnostic 

tests 


A test used to disoover areas of strength 
and weakness in the attainment of the 
expected outcomes of instruction. 
Synonymous i Placement test. 


Difficulty 

index* 


See Item analysis. 


M sorimination 
coefficient! 


See Item analysis. 


Facility index* 


See Item analysis 


Srid* 

Item* 


See Table of specifications, 

A question, an incomplete- sentence or other 
task in assessment designed to elicit a 
response from the examinee. 


O 
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Item analysis! 


A process by which the results of an 
examination are analysed for the purpose of 
identifying suitable items in terms of 
their difficulty and discriminating power. 
Difficulty index is usually expressed ae a 
percentage of the population answering the 
item satisfactorily* It is also called 

facility value, Th§ discriminating 

power (or coefficient) of an item is a 
measure of how well the item helps to 
differentiate between "good" and "poor" 
pupils, '’good’ 1 and "poor" being determined 
on the basis of performance on the entire 
test. 


Moderati on ; 


. A process in which items in a draft of a 
test or a grade obtained by assessment are 
critically scrutilised by a group of compe- 
tent persons to ensure that all the speci- 
fications demanded are met. 


'CM level; 


General Certificate of Mduoatiori at Ordinary 
Level - an examination taken at the end of 
a five-year secondary school course (aged 
16 years usually) . 


Objectives s 


The goals or aims of instruction or teach- 
ing stated explicitly (or sometimes only 
implicitly) in functional or behavioural 
terms* There may be several kinds, of ■ 

which content objectives and behavioural 
objectives are the most widely used* 


Placement test; 


See Diagnostic test. 


Pre-tests 


A process of administering a try-out form 
of a test for the purpose of obtaining 
experimental data to improve the final 
f orm * 


Psychomotor 
activity : 


Co-oi'dination of a mental process and a 
motor activity. 


Raw scores 


An empirical score obtained by direct 
application of a marking scheme or key. 


Scheme of 

studies s 


An expanded syllabus outlining the contents 
and indicating the relative emphasis of 
time to be devoted to each division or 
sub-division of the subject mat ter . 
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Shredding: 



Syllabus : 



Table of 

specifications : behavioural objects. 

Synonymous : Two-dimensional analysis , Grid, 

A scheme of studies explicit to include 
helpful methods and techniques of instruc- 
tions, teaching aids and references to 
source materials. 

Synonymous : Syllabus of instructions * 

Two-dimensional See Table of specifications, 
analyses : 



Teachers f 
manual : 



The use of certain criteria to analyse the 
function of a question with a view to 
re-writing the question in a most suitable 
form . 

A document compiled by a curriculum authority, 
specifying the major divisions and sub- 
divisions of the subject matter to be 
covered in the course, 

A diagramniatio analysis of the content and 
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Alias , J. et al. - Theoretical Constructs in Curriculum 

Development and Evaluation * Ministry of Education , 
Colombo, Ceylon, 1967? 24 PP- 

Alles, J* - Rotes on Structural and Functional Aspects of 
an Educational System relevant to Educational 
Adxnini s trat i on ; Ministry of Education, Colombo, 
Ceylon, 1967, 55 PP- 

Ashford, T * A * - Contribution of the ACS Examination Committee 
to Chemical Education* Journal of Chemical Educa- 
tion, Volume 42, 1965? P* 496* 

Bloom, B.S. (ed.) - Taxonomy of Educational Objectives? 

Handbook I, Cognitive Domain* Longmans, Green and 
Co*, New York, 1956, 207 PF * 

Bruner, J.3. - The Process of Education* Harvard U*P. f 
Cambridge, Mass*, i960, 97 p p* 

Examinations Committee ACS - National Standardisation Tests 
in Chemistry at all levels - High School, College 
and Graduate Level, 1955 to date, 

(May be obtained from Examinations Committee AOS, 
University of Soiith Florida, Tampa, Florida 53620*) 

Furst, E.J. - Construction Evaluation Instruments; Longmans , 
Green and Go*, New York, 195&, 334 PP- 

Goodlad, J.I. - The Changing School Curriculum ; Fund for 
the Advancement of Education, New York, 1966* 

IUPAG - The Effect of Examinations in Determining Chemistry 
Curricula (Mathews), Butterworths , 1967* 

Krathwohl , B.R*, Bloom, B.S*, and Mssia, B*B* * Taxonomy of 
Educational Objectives? Handbook XI, Affective 
Domain; David McKay Co*, New York, 1964? 196 pp* 
The second part of the work listed above (see 
Bloom) * 

Lindquist, fi.F, (ed.) - Educational Measurement; American 

Council on Education, Washington D.C., 1966 (sixth 
printing), 819 pp. 
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Ministry of Education (Department of Examinations}, Ceylon - 
Amended Syllabuses in Science Subjects, Ceylon 
General Certificate of Education (Ordinary Level) 
Examination; 1965 (includes sample question papers ) » 

Schools Council Publications, H.M.S.G., London - 

1 , Examinations Bulletins, particularly Nos, 1, 5* 

4 and 8 , 

2* Working Papers (all available)* 

The Schools Council - Educational Implications of Social and 
Economic Change, H*M*S.O. y 1967 * 

Taha, H* - Curriculum Developments Theory and Practice r 

Earoourt , Brace and World, New York, 1962 , 5^6 pp . 

Une sco — Meeting of Experts on the Curriculum of General Edu- 
cations Final Report, Moscow, January 1968; Unesco, 
Paris, June 1968, 29 pp# 
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II SUPPORTING EVIDENCE 



I # The Educational System of Ceylon 

from the point of view of science education 

by fi»F. Hal li well 

What follows 1 b generalised, sure to be inaccurate in a 
number of specific cases, but gives an adequately accurate 
working picture. The Ceylonese educational system is 
closer in its structure to that of the U.K. than to that of 
many other European countries or to that of the U.S.A. 

The main points of importance can be communicated by two 
figures and accompanying notes. 
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Figure 1 Structure of school population in Ceylon (1967) showing 

the population context of the programmes in General Science 
and in Chemistry, Physios and Biology for G-level. and A-level examina- 
tions (numbers approximate and rounded from statistic© supplied by the 
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TOTAL NUMBER OF CANDIDATES 
(first time and resit) 




Figure 3 Number of pupils taking O-level Chemistry in 

successive years (numbers rounded from statistics 
supplied by the Ministry of Education}. 

A and E mark the initiation of the General Science programme 
and the O-level Chemistry programme respectively * 



O 

ERIC 



Notes on Figures 1 and 2 

(i) The Ministry of Education and Science in Schools . 

Although all state education, at all levels including 
university level, is free, there are some private 
schools (either fee charging or free). Some of 
the latter are well equipped, have good library 
facilities, are of long standing and are held in high 
esteem. Nevertheless, all pupils staying at school 
after the statutory leaving age of do so, at 

whatever school they may be, with the intention of 
sitting for the Ceylon General Certificate of Educa- 
tion in a range of subjects - and the authority for 
the establi shmen t of syllabuses, the setting of the 
papers, the marking of the scripts and the publish- 
ing of the results resides with the Ministry* To 
this must be added the fact that the provision and 
organising of in-service courses to help teachers 
meet any changes in syllabus also lies entirely with 
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th® Ministry, In Ceylon, the considerable movement 
for reform in science teaching has also come through 
the Ministry, namely from the Deputy Director General 
for Secondary Education and his staff. 

In planning, especially in planning work for th# 
senior pupils, the closest co-operation with the 
country* s leading scientists is regarded as top 
priority* Nevertheless, in schools, in-service 
training and assistance together with the emergence of 
new patterns for examinations in th© Ceylon General 
Certificate of Education are the major vehicles for 
action, 

(ii) The Ceylon General Certificate of Education . This 

examination, like its counterpart in many countries in 
Asia and Africa, has evolved from, and still strongly 
resembles, the General Certificate of Education of the 
H,K* In Ceylon it is held in December. It can be 
taken at two levels? at the Ordinary Level at the age 
of 15+ or 16+| and at the Advanced Level at the age 
of 17+ or 18+ * These examinations are nearly always 

referred to as the G-level and the A-level examinations 

The O-level papers are set and moderated under the 
direction of the Commissioner of Examinations by a 
combined group which consists of university members, 
staff of the Deputy Director General for Secondary 
Education and members of staff from teacher training 
institutions. The marking of the scripts is done 
centrally in the Ministry * Much further increase of 
the number of scripts shown in Figure 2 will obviously 
lead to major administrative difficulties* In this 
connection it must be remembered that the examination 
in each science subject consists of two papers (one 
multiple choice of 5/4 hour and one free response of 
2 hours). Moreover, answers may be in Sinhalese or 
Tamil or English, although each candidate must use one 
language only. 

The A-level papers are set by invited university staff 
and an invited senior university member acts as 
moderator. The scripts, ia this case about l/lO the 
number of O-level scripts, are marked on a paid, but 
voluntary, basis by university staff. 

(iii) The pupils 

What is the background and intellectual calibre of the 
pupils taking the examination papers? In seeking an 
answer on© becomes aware that the pupils taking science 
are influenced by two pressures* selectivity by the 
educational system and competition from their contem- 
poraries. 
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Competition can tie dealt with briefly but its import- 
ance must not be under-rated , It arises from wide- 
spread unemployment , The fact is that if a boy 
obtains his 0 -levels and then his A-levels in sciense 
subjects, then, whether he goes to a university or 
other institution of higher education or not, there 
are more opportunities for him to gat a job. If he 
obtains Q-Ievel and even A-level passes in arts sub- 
jects he ©ay well find himself unemployed unless he 
has strong family influence backing him. 

To do well in science is, therefore, of importance in 
a contest which is much wider than an, educational one. 

Selectivity by the educational authority also becomes 
increasingly severe as the pupil advances. To get 
into an 0-level class the pupil must have don© well 
in his mathematics, in general science and in his 
first language (Sinhalese or Tamil, or occasionally 
English), Parental wish can also play a part at this 
stage, and as can be seen from Figure 1 about I 0 f« of 
the country's age group joined 0— level science classes 
in 1967 . Because the selection is partly on academic 
ability and partly on parental wishes, there is a 
fairly wide range of ability at the 0-level examination 

To go into an A-level science class, however, a pupil 
must have passed the 0-level examination in his first 
language, in mathematics, and in four othor subjects. 
Further, among these six passes there must be three 
which he has passed at credit level — two of which 
must b© in the science subjects he proposes to take at 
A-level, This high standard for entry into the 
A-level class is one of the reasons why such a high 
proportion of those taking 0-level re-take the examin- 
ation a year later. The other reason is that if they 
have failed first time they try to pass on the second 
occasion for the economic— sociological reasons given 
above. 

To get into a university a pupil must achieve at least 
5 credits (not just passes) in his four A-level sub- 
jects and also have reached an arbitrarily set level 
in the aggregate of marks for all four subjects. 

These requirements must be set against a background of 
provision for learning. In 0 -level classes a pupil 
has 5 x 40 minute parir-de per week in each science for 
two years, and at end he must take papers in all 

three sciences ( phy bd ps * fchemistry , biology) and in 
mathematics. In A-lv-.f&X classes, thp time allowance 
averages 8 x 40 minute periods per week in each of his 
four A-level subjects - one third to one half of this 
time being spent in a laboratory. The us.ual combiai'.i- 
tion of subjects for A-level ares physics, chemistry. 
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pur© mathematics and applied mathematics , or chemistry, 
botany, geology and physics* 

In X966 f 53% of those sitting G-ievel chemistry passed, 
and 22% obtained a credit* About 15% did well enough 
in their chemistry and other subjects to enter A-level 
classes . 

( iv) The teachers 

Teachers of A-Xevel classes are all graduates in science 
subjects* Some will have had professional education 
training* 

Teachers of O-level classes are, by a large majority, 
non-graduates, but have been professionally trained 
at one of the two Ceylonese teacher training colleges. 

The Ministry, however, has an extensive in-service 
training and aiding programme. Although each year 
there is usually only one vacation course in science 
lasting 2-4 weeks, there is a well established system 
of one-day study circles taking place at weekends under 
the direction of the staff of th© Deputy Director 
General for Secondary Education. These study circles 
are held all over the island, number about 40-50 in a 
year, and have been running now for four years* This 
means there i© a very close link between teachers on 
their home ground and those in the -Ministry responsible 
for planning and implementing new schemes* 

( v) Some special problems 

One of these is common to many countries but perhaps 
particularly to Asian countries* Centuries of trad- 
ition have held that the function of the pupil was to 
learn at the feet of the teacher whose business it was 
to know* Any new relationship between pupil and 
teacher which in any. implies that the teacher cannot 
supply th© answer for the child to learn is regarded 
with suspicion amounting in some cases almost to con- 
tempt by many parents and pupils, and even teachers* 
There is thus an inherent resistance t© new modes of 
learning based on co-operative exploration and it is 
certainly necessary for any new proposals to be 
accompanied by extensive guidance for teachers! 

But perhaps one of the great problem© in Ceylon r s 
education system is that of language, a problem also 
mat in some European and other Asian countries* The 
language of pupils in Ceylon Is approximately i 

75% Sinhalese 
20^ Tauil 
5^> English 
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But the special and crucial point is that an the whole 
neither group can spent the other native language. 

Very few,' even educated, Ceylonese can converse fluent- 
ly with a countryman in the other native tongue? 
communication between Tamils and Sinhalese is predom- 
inantly through English, It is not without signifi- 
cance that the first version of the new chemistry pro- 
gramma is printed in English although Sinhalese and 
Tamil versions are being prepared; the problem arises, 
however, of new technical words in the native language . 
Examination papers have to be printed in three lang- 
uages, but those marking one set of scripts on the 
whole could not mark the other. The- situation with 
respect to textbooks ie not an easy one, especially at 
A-level, for the cost of producing a book that can 
only be read by a small group within the island and by 
no one outside it is vary high. Nevertheless progress 
has already been made, as will be discussed in the 
second section of this Working Paper. 

Lastly, any country will be confronted with the cost of 
equipment, and Ceylon's educational budget in absolute 
terms is not big. In general science, where the 
number of classes and of pupils is high, improvisation 
is essential, At O-lavel, improvisation ia still 
necessary, but special apparatus and chemicals are 
available as required. The policy at A-level is that 
what is necessary for sound learning must be supplied - 
but reform at A-level is only just being-planned. 

On the other hand it must be realised that sclent ; I o 
equipment and chemicals, at least at the beginning of 
the task of coping with the development, need to be 
purchased from abroad (Europe is the chief source) and 
transported by sea. This means the ua® of foreign 
currency and delays in delivery. Unless the item 
required is held in stock there may be up to a year’s 
delay in a school’s obtaining it, a situation aggravated 
by the closing of the Sues Canal. The Ministry has 
begun a scheme for producing locally must of the scien- 
tific equipment needed by schools* About 18 miles out * 
of Colombo, building has begun on a site for an Insti- 
tute of Science Education. Part of the site is being 
developed as a residential centre for training science 
teachers and for housing curriculum reform groups. 

Part of the site comprises drawing office, production 
workshops equipped with modern machinery and a distri- 
bution centre for apparatuc designed and tested in the 
other part. This production centre is already in 
initial stages of operation. Such an imaginative step, 
possibly vital to Ceylon’s development of science 
education - especially if there is to be a big increase 
in the number of pupils in O-level and A-level classes - 
is almost certain to clash with business interests of 
foreign suppliers . 
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Soag aspects of the development of the new schemes for trash - 
ing chemistry In Ceylon 

What follows ia not an outline of the history of the 
Curriculum Reform Project, out a series of comments on some of 
its problems and some of its successes and failures, attention 
to which is thought likely to be useful to any group involved 
in a similar effort, A project focussing on O-level work 
will have a different set of problems from one concerned with 
A-level. The latter deals with a group of pupils who are 
more mature and of a narrow range of high ability. Moreover, 
the content is much more sophisticated and, for many pupils, 
is transitional to university work. The problems of A-level 
work are chiefly concerned with establishing, both in the , 
field of planning and in the area of in-service teacher 
training, close and fruitful co-operation between university 
world and school world, Ceylon is only just embsxking on i cs 
A-level project. This part of the paper is therefore limited 
to the O-level project, begun in 1961 and now in its third 
year of trial in schools on a scale large. enough to be exam- 
ined nationally. 



It is convenient to consider the problems in the frame- 
work of the four steps which will need to be taken s 

1. Formulating the educational AIMS of the Project, 

2. Planning and carrying out lines of ACTION thought 
to be necessary to achieve the aims. 

3 . Setting up machinery for valid and reliable ASSESS- 
MENTS of the action in the light of the aims. 

4 . Establishing permanent machinery for RE-APPRAISAL 
on the basis of feed-back and th® results of 
assessment, 

AIMS* These are formulated at different levels. There axe 
the general, that is politico-sociological , statements 
of intentions made by Ministers of Education and the highest 
directorate level of administrative services | there are the 
directions given to teachers in terms of programmes for schools} 
there are the intentions (often imprsalsely formulated) of 
teachers in the classroom 1 there are the definite demands made 
on pupils in examination©. That this translation of objec- 
tives from general ter®® through to pupil behaviour should be 
checked for internal consistency might well b© thought to be an 
obvious necessity. It is not commonly done explicitly. "©w 
far it has been done in Ceylon may be judged from what follows. 

The general aims that were behind the initiation of 
the General Science Project in 1957 and of the O-level Specific 
Sciences begun In 1961 were also stated in the report of the 
National Commission on Technical Education, July 19o3» 

Sessional Paper X of 1963. 
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In. a develop! as = cun try, persons having the 
necessary aptituuaa and ability enould ha 
selected and tr^Insd to Make fullest 

contribution to the various fields of. industry, 
agriculture and coreiaeroe. The purpose ei 
education is not merely to mould the person- 
ality of the student in order -so fit mm in o 
the society in which he lives, but els° *<> 
develop his innate talents and skills, a 
that he will be an asset to the nation. 

Technical education merits special attenUo 
in view of its vital relationship to national 
development . 

The education provided at present is 
academic, and is influenced largely by the 
requirements for entry to a university. J-h 
World Bank Mission has drawn attention to the 
fact that in the family the children, especially 
among the more prosperous classes, are •*** _ 

iveiv sheltered. Initiative, self-confidence, 
experimentation, inquiring habits of »i£d, *** 

not encouraged from early childhood, 
tendency in education has been to stress th- 
encyclopaedic and purely academic aspect of 
education ■, rather than skill# and independent 
a^lutio*) of new problems. It k * 8 J’'*? d th 
oreate a prejudice against manual work and the 
acquisition much ne®4«d skill# t ■»* 
thus obstructed economic development,, Book 
learning and memory rather than reasoning 
ability have been stressed, even in village 
schools, where children com© frora predominantly 
a|pr A cultural eQfiiffiUUitiaa * 

. subject stage of translation, now directed much more 
at teaohers, we findi 

«The pregnant uooial changes that had taken 
dace in Ceylon, some of whioh have been indloa- 
tirlbo?., Sade’it clear that their effects - 
the changes in the nature, composition^ aspira- 
tion and function of the S.C.E, 0 Level 
population - demanded a change in the educational 
objectives hitherto assumed - objectives^ which 
involved not only the knowledge and skill compon- 
ents of education, but also the component of 
attitudes.'* 

(Ceylon Science Teaching Bfvelopment 

Programme | a report prepared for 

Uneaoo , department of Education, 

Ceylon, 1363.) 
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These directives now in the form of questions are repeated 
in slightly altered form in the detailed printed version of the 
O-level Chemistry Schemes (Syllabuses of Introduction and 
Schemes of Work in Science Subjects for G.C.E, Classes, Depart- 
ment of Education, Ceylon), 

Nevertheless it is pertinent to ask for operational defi- 
nitions of some of the phrases used. For example, what is it 
that a ’’scientifically literate citizen" can do that distin- 
guishes him from a non-scientifically literate citizen? 

Until this is done it is doubtful whether we have any 
basis for judging whether the examinations which pupils have 
to take can find out whether they are scientifically literate 
or not. 

Lastly, the Specifications for Examinations have been 
formulated by the Ceylonese Ministry, Here the influence of 
"The Taxonomy of Educational Objectives", Bloom et al, (New York, 
I. 9 . 56 ) is acknowledged. The Specifications are stated in the 
report prepared for Unesco (supra). 

The details of the way these specifications are used to 
plan an examination paper, and the statistical methods by which 
the validity and reliability of the assessment is checked, are 
given in Part III - Background Papers, 2 and 3, This aspect 
of the work has been done with great care and it is possible 
therefore for the Ceylonese Ministry and ' for outside observers 
to judge to what extent the objectives are being achieved. 
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OBJECTIVES 1, 2, g, 3*, 4 
An Explanatory Mote 



In this analysis objectives are classified using a 
modified form of "The Taxonomy of Educational Objectives", 
designed by Bloom et al. for the Cognitive Domain (Bloom, 
Benjamin S,, et al., 1956 i A Taxonomy of Educational 
Objectives; Longmans Green and Go., New York). 

1. This category includes test items which require a 
response involving primarily the RECALL of specific 
fasts , terminology, conventions, trends, classifica- 
tions criteria, methodology, principles, generalisa- 
tions, theories, etc, 

2, Test items in this category demand a response calling 
for more than simple recall. An element of 
COMPREHENSION is required and may be of the following 
types 

translation of mathematical verbal material 
into symbolic statements and vice versa, 

interpretation of data, statements, etc. 

(including simple explanations, summaries, 
etc . ) , 

extrapolation from data, statements, etc, ■ 

(i.e. extension of trends beyond the given 
data, determination of effects, corollaries, 
etc , 

3, Test items in. this category will demand responses 
involving the APPLICATION of abstractions in. parti- 
cular and concrete situations. The abstractions may 
be in the form of general ideas, rules of procedure, 
general methods, technical principles, theories, etc. 

5', These test items will require responses such as in 3, 
but will include numerical CALCULATION, 

4. Test items in this category will include items usually 
classified in Bloom 4, 5 and 6, and will involve 
responses calling for ANALYSIS, SYNTHESIS and 
EVALUATION. 

Among others this will include the followingi- 

reeognition of explicit and implicit relation- 
ships, structure , etc., recognition of unstated 
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assumptions, discriminations of facta from hypo- 
theses', organisation of ideas, statements, data, 
etc,, proposition of methods of tasting hypotheses 
theories, etc,, design of simple experiments, 
recognition of logical accuracy, consistency, etc, 
comparison of theories, generalisations, etc. 
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AC Li An The .ver;«& .ituei: to bring a 3chn-.y Into aemg^in 

r*. r;i^ ■ jrCC.T: hive I eughlv iCiiO^KL t./:G I Oii.O 'AiirJ 1 1 no 2 

rlarmir.g and decision making, and estaoli suing 
a .fairly penxauent ref onii group ; 

2. ,v r i ting, c.-Ueci-ing f eed-btiu* , and ro -correcting 
o .f d & t s 1 1 a ■ i & o h em e s o x v;o rk ; 

the settings up of extensive and continued 
in-yervice and predsrvice teacher training; 

4* trio provi si on of adequate ..earning situ a — 
t i o r, a an o r e a ;> u r c o a ; 

5* L-h 0 e s tab 1 i s ?iTB e n t of a v a I x G and reliable 

instrument for assessing pupi Is ? achievement. 

What lias the Ceylonese group done about this? 

The Jciar.ee Teaching Project has been centred in the 
Ministry under the direct and extensive guidance of the deputy 
Lix'ee tor General for Gecondary Education. He and the staff 
concerned have had close links vnth similar work in other 
countries either by going to these countries to work intimately 
with the proposals or by the bringing of overseas experts to 
Ceylon. The zxtont of the careful look at other people's 
efforts can be judged by trie following facts . Two of the staff 
8 d e n t one year in the U».v* (another three opending six weeks 
there); sixteen, spent one year in the U.b.A* arid two spent 
short periods in .India and Australia; two 5 closely associates 
with the On err* 1 ;x try Project, spent a year in Bangkok with the 
Unesco Chemistry Teaching Pilot Project for South-Last Asia, 
and one of these has continued this close association with this 
Asian project fer a second year. 



In the reverse way, groups of leading science teachers 
ti 4 A , were invited to take part in inspectorate bis- 



from the - - * 

cussions and teacher- training courses , and leading figures 
associated with reform in the U.S*A. and Australia have been 
invited to Ceyloi* , 



In 

modern trends 
into details 
basis for the 
writing of a 
further trial 
distribution 
back lasting 



this way the Ceylonese group were well informed of 
in other countries and had time to translate these 
which were appropriate to their own needs. The 
preparation of detailed schemes of work was the 
preliminary draft, trial and feed-back, rewriting, 
and feed-back and finally extensive printing and 
followed by an intensive system of trial and feed- 
over lif years. 



Such work is now being continued by the science' super- 
visors in consultation with the writing group- Because the 
writing group had so much contact with movements in other coun- 
tries, were themselves experienced teachers and had already 
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formulated a set of objectives which acted ao criteria for 
selection or reje otion, they ware able to write a draft scheme 
that had programmed learning as a b&eis and was essentially 
viable. "‘The feed-back was limited to begin with. Groups 
of six or seven teachers rare asked to try out small sections, 
and more extensive areas were tried out with teachers at 
vacation courses. The latter were very valuable in that not 
only was feed-back obtained, but the teachers themselves were 
introduced to and participated in the new work that was yet , 
to come. 



After the writing of the second draft there were 
more extensive trials lasting over one year.” Now it was 
arranged that a rang© of types of schools and pupils was 
covered, the writing group themselves going into schools and 
teaching as full time teachers . 

During the vacations, discussions lasting one or two 
weeks were held with selected senior science teachers • 

Finally a printed version was published along with 
H Amended Syllabuses in Science Subjects" (Department of 
Examinations SA 6j6(Sc), 1964)* Following this, very exten- 
sive in-service trials were held by the working group every 
weekend for 1^ years . In this way in the course of a year 
almost all teachers in the island were contacted. The dedi- 
cation necessary and the amount of labour, unrelieved by any 
break, which was involved is obvious to say the least. 

As a result of such interchange of experience, certain 
areas of the printed scheme are now under revision. The 
setting up of in-service training has been covered in commenting 
on the writing and collection of feed-back - in fact these two 
aspects are not separable. Pre-servic© training which supplies 
the majority of (non-graduate) teacher© for G-level work is 
under a separate branch of the Ministry* The work of teacher 
training and of curriculum reform should be, closely oo-operativ© . 
In Ceylon the two are not yet as closely knit as they are in 
some other countries. 

In turning to the setting up of adequate learning 
situations for science one is chiefly concerned with finance; 
supplying what is needed by a school (and conversely seeing 
that what is supplied is in fact needed) and building labora- 
tories. 

Ten years ago there were about 166 school labora- 
tories in the island, chiefly in the old-established and 
private schools. Now there are about 700. . Ten years ago 
in most schools science teaching was chiefly a matter of use 
of blackboard and a few demonstrations * Now the inspectorate 
report that in most schools pupils ars doing experiments at 
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least once a week* J.n 
been set up a centralis 
requesting supplies, an 
unit. The cost of t'ni 
of buildings, is seen f 
schools . 



the last ten years, too, there has 
ed store of equipment, a. system of 
d a centralised repair and workshop 
s change, quite apart from the cost 
rom the following figures for state 



1957 

1963 

1967 



Equipment 


Chemi. 0 


a Is 


156,000 Rs 


52,000 


Rs 


4 00 ,000 Rs 


172*426 


R s 


400,000 It® 


325,605 


RS 



A list of minimum equipment 



has been prepared by the Ministry* 



There are about 150 laboratories in which 
O— level and A,— level chemistry is done. These have good faci- 
lities and usually one or one and a half laboratory assistants 
to each laboratory* General science is not taught in these 
rooms. There are 4 GQ- 5 GO laboratories, less expensively . 
equipped, which are used for 0-level work, only , sometimes lor 
two subjects. Most of these have electricity or gas produced 
on the premises, Running water is not available in many 
schools because of the expense of piping it, but much of the 
work at G~level can be done with a static supply and a system 
of syphoning. 



There is one ARISBA laboratory in use, erected 
by the U.N, School Building Unit as a prototype for Asia. 

This has static water, storage cells and portable arrange- 
ment for heating, operationally planned, arrangement of 
working places, and. storage facilities* 

The question of books involves the language prob- 
lem mentioned before but at the same time provides for the 
evidence (along with the type of examination questions pupils 
now have to answer, and together with the increased amount of 
experimental work) that the new schemes are not just on papers 
a differently trained group of young people is emerging. 

In the 1940a the principal textbooks in use for 
0-level in the island were Littler: Elementary Chemistry, and 
Kees School Chemistry. 

In the 1950a the popular textbook was Wilkins t 
Fundamental Chemistry (translated into Sinhalese and Tamil in 

1956). 

In the 1960s Holderness and Lambert's School 
Chemistry has been predominantly in use. 

But the fact is that with the change of programme 
and type of examination question it is difficult to answer the 



€8 



FILMED FROM BEST AVAILABLE COPY 



- 63 - 

new type of demands by reading only the book© quoted above. 

Even the content of the new schemes is not covered by those 
more traditional texts. 

The Ministry has therefore had to have new 
textbooks written and this task has been undertaken by those 
responsible for the reforms in science education. Unfortun- 
ately these new texts are only published in Sinhalese or Tamil. 
In the opinion of the compiler of this paper, these texts should 
be translated into English so that those who are considering 
writing textbooks for their own country can benefit by seeing 
how ideas from many other parts of the world can be woven into 
something that is relevant to ones own country. 

The characteristics of these new text books are 
the completely fresh look at content and approach, their 
inclusion of much of interest from schemes in other countries, 
the deliberate building into the text on almost every page of 
little answers and questions to the pupil reader and suggest- 
ions as to what he might do at home, and (unusual in a "text- 
boQk n ) chapters on the method of solving problems by invest!^ 
gation, weighing of evidence for hypotheses and putting forward 
explanations. These chapters take the process of being scien- 
tific away from classroom and school into the village, the home, 
the world of newspapers and personal argument, and then bring 
it back again to the immediate problem of the chemistry lesson. 
There is a deliberate attempt here to make the phrase "scien- 
tifically literate" take on an operational meaning. It is a 
bold and imaginative piece of production, yet all the time it 
has its feet firmly on the ground. 

ASSESSMENT! There are two separate areas where a valid and 

reliable method of assessment is necessary. 

The first is in the realms of the Ministry’s own 
activities. It laid down its objectives clearly. Are these 
objectives being reached? Some of the evidence w^ll be 
qualitative* the different kind of examination question, the 
increase in laboratory work in schools , the demand for a new 
type of textbook. This will give enough indication for further 
action to be taken, but a detailed statistical assessment of 
the success of the policy is a highly specialised research 
project and is not available at the time of writing thin Working 
Paper. : . . 

The second area is that of pupils ' activities* 

The instrument for this is usually a public examination. In 
Ceylon this instrument has been subject to careful scrutiny 
over seven or eight years. The analysis of papers for content 
validity and objective validity, the statistical analysis that 
has been done on the results, are outlined in Part III - Back- 
ground Papers, 2 and 3# 
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2 , Design for Setting 

the Sample Question Paper in Chemistry 
for Higher Secondary School 
Leaving Certificate Examination 

NCERTj India 



In order to eliminate the wild role which subjectivity 
plays in setting, answering and scoring questions, to ensure 
a better coverage of objectives, a more effective distribution 
of questions over the whole syllabus, inclusion of a variety 
of questions and thereby to do away with selective study and 
cramming on the part of pupils, the following design for 
setting the question paper in chemistry is suggested. 

The design ia only a suggestion, and leaves scope for 
necessary alterations according to the needs and nature of 
the subject matter. The suggestions are intended to be 
followed in spirit and the weightages proposed to different 
items may be considered as only approximate- The setter can 
at his discretion, there tore, feel free to make changes in 
respect of the weightages, if necessary. 

X Weightage to objectives 

In order that the questions framed may not be based on 
just one or a few objectives, it is also imperative to deter-* * 
miue the freightage to each objective and thereby distribute 
questions over all the objectives in proportion to their 
importance- 

In view of the foregoing considerations, a scheme for 
the freightage to different objectives for this paper is 
suggested below. 



TABLE I 

Weightage to Objectives 



S, No. 


Ins true ti anal objectives 


Marks 


Percentage 


1 


Knowledge 


48 


48 


2 


Understanding 


57 


37 


5 


Application 


15 


15 




TOTAL 


100 


100 
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II Weigfatage to content areas 

Towards obtaining a wider and effective coverage of the 
syllabus it is suggested that an earnest attempt be made to 
distribute the marks alloted over different parts of the 
syllabus , 



TABLE II 

Distribution of Marks over Major Areas of Content 







Total 


Marks 


S * No , 


Areas of Content 


Major areas 
of content 


Syllabus 

Divisions 

in 

Question 

Paper 


i 


Laboratory process© s 


2 


2 


2 


Chemical and physical changes 


8 


8 


5 


Atomic weight, molecular 
weight, laws, etc. 


6 


6 


4 


Solutions 


5 


3 


5 


Halogens 


6 


6 


6 


Acids, bases and salts 


8 


8 


'7 


Nitrogen and phosphorus 


10 


10 


8 


Sulphur 


10 


10 


9 


Na, Ca , Mg and Si 


7 


7 


10 


Metals 


20 


20 


11 


Carbon and hydrocarbons 


15 


15 


12 


Qualitative analysis 


5 


, 5 




TOTAL 


100 


100 


III We 


rightage to forms of Questions 



It is common experience that the inclusion of only a few 
essay questions in the question papers encourage guess-work 
and cramming of set answers to selected questions. It is 
therefore suggested that the question paper may* include all 
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trie four types of questions * viz* the essay type, the short 
answer type, the very short answer type and the objective 
type . 

A suggested weight age to different forms of questions is 
given below* 



TABLE III 

Weigh tage to Different Forms of Questions 



i 

S,* Ho. 


Forms of questions 


do* of 
questions ; 


Marks 


Percen- 

tage 


. 1 


Essay 


■ 5 


25 


25 


2 


Short answer 


25 


50 


50 


5 


Objective 


25 


25 


25 j 




TOTAL 


55 


100 


100 



difficulty level, language and arrangement of questions 

One of the important functions of examinations is to dis- 
criminate between the low achievers and the higher achievers t 
A question paper is considered to be a good paper if it dis- 
criminates well. For this purpose it is recommended that 
the question papers may contain in due proportion questions 
of various difficulty levels ranging from easy to difficult* 

I t may , however, be ensured that they are not too easy nor 
too difficult for the level* 

The wording of the questions must be suf fioianjbly clear, 
explicit and unambiguous for each examinee to comprehend the 
questions in the same way as every other examinee and to be 
able to know precisely what he is expected to write or draw 
and of how great a length. 

From the psychological point of view and in order- to 
create confidence in the examinee, the questions as far as 
possible should, be arranged according to the increasing level 
of difficulty. This arrangement may be done within the 
topic or ..the content also if necessary* 




V Scheme of option 

Options as such have no place in a schema of good examin- 
ations, -.and hence no option is provided in this paper. 

i * 

• i • ■ . 
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V I Scheme of sections and their admini Stratton 

The question paper may he divided into two sections * 
section, A may Include objective type and very short answer 
questions; Section B may include essay and short answer 
type questions. 

It is suggested that both the sections he administered 
simultaneously with clear instructions to- the examinees that 
the answer papers for Section A will be collected promptly 
at the end of half an hour* After the commencement of the 
paper, the students will have to be instructed in advance 
that if they finish Section A before time is called* they 
may go right on to Section B without awaiting further signal- 
The time allotted to each section along with other subsidiary 
details is suggested below. 



TABLE IV 

Time Allotment to Sections 



Sections of 
the sample 
question 
paper 


Form of 
question 


Total 
mark a 


No. of 
questions 
to be 
attempted 


No. of 
questions 
to be set 


Estimated 
time for 
answering 
in minutes 


Section A 


Objective 


25 


25 


25 


30 


Section B 


Essay and 
short 
an rawer 


75 


50 


50 ' 


150 



VII Directions 



Adequate directions regarding the number of questions to 
be attempted* allotment of time to different sections and 
other necessary instructions may be given clearly in the 
Question paper, 

VI II Model answers » scoring key and marking scheme 

It is suggested that model answers and marking schemes 
should be prepared along with the question papers. This will 
increase uniformity In scoring and will reduce to a great 
extent the chance of inclusion of vague questions requiring 
answers of an undefined scope- This will also check possible 
errors (numerical and others) that sometimes unwittingly creep 
in. This suggestion should, therefore, be invariably adhered 
to . 



O 
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The design suggested here is only a first step towards 
improving examinations. It may require modifications after 
it is given a trial for two or thres years. 
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SAMPLE QUESTION PAPER 
i * Blueprint 





* - - ■ 

..bJECTIVE 


hN QW LEDGE 


UN DEE- 
'S TAM DING 


APPLICATION 


S.KIL 


. T 


» vr* 1 

^ ; t , 

— - ----» • 4 




CONTENT 


E 


s 


0 


£ 


s 


0 


E 


s 


0 


rp 

.u 


Q 


0 


j 


j lab oratory 
f processes 


* . 


* * 


p * 


* # 




1(1) 


* * 


ft ft 


1(1) 


—4 






£ i r. ■ : 


' 

j 


Chemical 
and phys- 
ical pro- 
perti es 


f * 


2(1) 


1(1) 




2(1) 


2(2} 


ft #? 


• # 


1(1) 


I 

t 

t 

= i 

i 






t ; ' ' ; 


j 

{ 

( 


Atomic 

weight* 

. molecular 
weight, 
laws, etc* 


# # 


4(2) 


m • 




. * 


ft * | 


* * 


2(1) 


1 # 








6(;.', I 


j 4 


Solutions 


* V 


« 9 


ft ft 


r r 


2(1) 


1(1) 


ft * 


ft • 


ft * 








5(2) 


Ll 


Halogens 


5(1) 


* # 


< ft 


ft ft 


ft » 


1(1) 


ft ft 


. * 


'ft * 








6(2) 


i 6 
! 


Acids j 
; bases j 
salts 


* ■ 


2 ( 1 ) 


id) 


ft ft 


2(1) 


i(i) ; 


• ft 


2(1) 


9 « 








8(5) 


7 


| Ni trogen 
and 

phosphorus 


5(1) 


p ■> 


i(i) 


ft • 


2(1) 


id) 


• * 


ft * 


KD 








10(5) 


a 


Sulphur 


5(1) 


2(1) 


ft ft 


ft • 


2(1) 


■ * ft 


• * 


ft ft 


1(1) 








10(4} 


9 


,a&, Ca, 

, Mg' and Si 


# • 


2(1) 


* V 


t # 


2(1) 


i(i) 


« • 


2(1) 


■ 4 








7(4) 


10 


Metals 


5(1) 


4(2) 


2(2) 


# * 


4(2) 


2(2) 


• * 


2(1) 


' 1(1) 








20(11 ) 


II 


Carbon and 
hydro- 
carbon 


5(1) 


2(1) 


* * 


* ■ 


6(5) 


2(2) 


ft * 


ft ft 


# P 








15(7) 


1? 


Qualitative 
an&ly sis 




ft ft 




ft ft 


2(1) 


KD 


* # 


I # « 


2(2) 








5(4) 




Sub-total 


25(5) 


18(9) 


5(5) 


ft ft 


24 ( 12 ) 


X\ 

H 

Hi ( 


ft ft ■ 


8(4) 


7(7) 










Total 


4 


48(19) 


57(25) 


15(11) 




100 ( 55 ) 



Lotei Figures within brackets indicate the number of questions and figures 
outside brackets indicate marks* 



Summary* Es oay (B) No* 5 Marks 25 

Short answer (S) No. 25 Marks 50 

/Objective type (0) No, 2 5 Marks 25 

Scheme of Sections t Section A * Objective questions 

Section B - Short answer and essay question 




V.F 
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3. Notes on the Assessment of Practical Work by Teachers ; 
' A* • Level Obaaistry, Nuffield Scheme, IT, C 

by J.C, Mathews 



The assessment of practical work by teachers has been 
undergoing trials since September 1966, Th® purpose of these 
notes is to explain this form of assessment to teachers who 
are about to start the Nuffield *A' level chemistry course in 



The advantages of assessment by teachers 

1. The assessment can be done by those who actually observe 
the students at work over a long period of time, so that prac- 
tical skills, abilities and aptitudes, which are not displayed 
in written work can be taken into account. In this way the 
objectives of the assessment can be made wider than those of a 
practical examination assessed on a single occasion on written 
evidence alone, 

2. Assessment on several occasions is necessary to cover 
adequately the variety of experiments and techniques which 
comprise the practical work of the course; and the effect of 
chance failure or success is less. 

jj. .The assessed practical work need not be limited to those 
eiperinento which oan be readily administered to a large number 
of candidates in a limited time. The restriction of practioal 
examinations to such experiments could have an undesirable 
effect on the variety of practical work in the schools taking 
the Nuffield course. The form of assessment set out in these 
notes should lead to a greater freedom of choice of practical 
work ±i fc_;..',ole. 

4* This scheme does give teachers the opportunity to become 
involved in the assessment of their students. It could lead 
to a greater awareness of the objectives of practioal work and 
a situation in which teachers and .-.aminera work together 
towards these objectives. 

The .outcome of the school trials 

i ‘ ! 

v,’ During the two years of trials, eight experiments, all of 
them part of the basic course, were chosen by the Project's 
Headquarters team for assessment'. Guidance was sent to 
teachers on the conduct of the experiments, . the objectives of 
each* the manner in which marks could be awarded. The 

results for each. student were entered on a record card and the 
record cards were, reviewed by the Examiners at the end of the 
first and second years. The main conclusions were as follows. 
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1, Teachers tended to use a restricted range of marks, and 
a candidate’s total mark «a s rarely less than the half mark. 
The mean mark was correspondingly high. From an analysis 
of the marks, coupled with the direct observation of practi- 
cal classes by the Headquarters Team, it can be inferred that 
the spread of ability is lees in practical work than it is in 
written work, and it is unlikely that a student can be maid 
to have failed in hin two year© of practical work, 

2, The objectives of practical work were the subject of 
much debate. Finally, all the teachers who took part in 
the trials were invited to state what they considered the 
objectives to be. The objectives and their weighting, which 
are set out later in these notes, follow closely the outcome 
of this inquiry. 

3. Many teachers felt that the detailed instructions, given 
by the Headquarters Team for each of the assessed experiments, 
unduly restricted them in. their assessment. It became clear 
that no single set of instructions would meet the wishes of 
all the teachers, and it was decided that the proper course 
was to give as much freedom as possible to the teachers in 
the choice of experiments for assessment and in the conduct 

of the assessment. Some uniformity is necessary, however, in 
the objectives, the areas of work, the type of experiment, and 
the form in which the results are recorded. 

4. It has been a constant policy of the trials that the 
assessment should be conducted on normal school, work and be 
as unobtrusive as possible, but often there was a noticeable 
difference in the attitude of a class to the work when they 
knew it was to be assessed. Some teachers welcomed the more 
diligent work, others regretted the more formal atmosphere, 

5. Teasers were rightly concerned lest the raw mark which 
they awarded their students would be the one used in the final 
•A» level mark. The raw marks were used by the Examiners to 
determine only the distribution of marks within a teaching set 
and this policy will continue, 

6, Possibly the most difficult decision which had to he 

made was whether there was auffioient difference in the general 
standard of practical work between teaching sets and between 
schools to call for the use of a moderating instrument in order 
to make adjustments to the marks. For the time being, it 
seems that some form of moderation is desirable, but the matter 
will be reviewed from year to year. 

I 

7* Teachers found that those objectives which could be ma rked 
from written evidence were easier to assess' than those which 
called for direct observation by the teacher during an experi- 
ment. ,It was found to be very difficult to assess more than 
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o no of the latter during 5 any one 
assessment by direct observation 
be cor © more f ami .ti ar wi th i t an d 
in which they know they can 'pick 
virtues without interfering with 



experiment: * I.t is hopea that 
will become easier as teachers 
can choose those experiments 
out particular faults and 
the norsal teaching work. 



The Form of Ass ggstnen t af ter 1 S&& 

T h e ob j e c tiy e s 

The following objectives* with weighting given as a per-* 
cent age, are suggested* 



1* Skill in observation (25/ 6 )- 

2* Ability to interpret obaervationg (15 /a). 

3- Ability to plan experiments ( 10'^} , 

4 * Manipulative skills ( yG'ft) . 

5* At titud.es to practical work (20 fo) . 

In many respects the form of assessment advocated in these 
notes has still to be evaluated, and it would be wrong to regard 
this statement of objectives and weighting as fixed for all time* 
Teachers may wish to modify both the objectives and the weighting* 
particularly in the finer divisions of the objectives. But for 
the time being it will help in the evaluation of this form of 
assessment if teachers can keep as close to the suggested 
pattern as possible. If teachers feel that theyi must deviate 
from this pattern they will still be asked to show on the 
record cards how they allocated marks to particular objectives. 

The objectives 1-5 provide the broad basis for the final 
assessment* but for assessment on particular occasions a finer 
division is necessary * The following notes on the objectives 
may be helpful* but teachers should not feel that they are rigidly 
bound by them, . ‘ * 

1 * Observation alone has little merit unless it is accurate 
and relevant* Its assessment leans heavily on a students, 
ability to report* and it is inevitable that most of it will 
take the form of awarding marke for written records of observa- 
tion. But there are two aspects of observation for which 
teachers can give cr#&it while a conventional practical examin- 
ation cannot* and it' is hoped that neither will be overlooked. 

The first ie oral reporting which is usually more immediate and 
less rehearsed than written reporting* The second is' a 
student's observation and interest in unusual and unexpected 
features of an experiment even though they are not relevant to 
the immediate purpose of the experiment* It will not be easy 
to put a mark on these two aspects but at least credit could be 
given in a periodic review and grading of each student's atti- 
tude to practical work. 
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2. The assessment of interpretation is similar to that of 
observation in that teachers can only judge what students write 
or say. It will give credit for an analytical approach to 
observations, including unexpected observations and the ability 
to eommen b critically on practical situations. 

£ Jt The ability to plan is most easily assessed by asking the 
students to write their plan and submit it for marking before 
they start, but this can only be done in those experiments for 
which inf ormation to students on procedure is restricted. It 
should include the ability to recognize sources of error, and 
the ability to make predictions about other experiments. 

It can be argued that both i nterpre tation and planning 
can be assessed on written examination papers. This may be 
so, but a student’s interpretation of observations which he 
has actually made and his plans for an experiment which he 
knows he must try to put into operation may well be different 
from his interpretation and planning of unreal situations on 
paper. The thinking processes may be similar but their con- 
text is different, therefore their outcome may be different. 

4* Mani putative skills can be assessed by marking the results 
of experiments, for example yield, purity, and accuracy (accu- 
racy of work rather than accuracy of calculation) ; they can 
also be assessed by direct observation of such things as 
orderliness, methodical working, dexterity and speed. .It is 
desirable that this assessment should not be confined to 
routine operations; it should include the ability of students 
to adapt their manipulative skills to new situations and to 
follow unfamiliar ine tractions . 

It may be possible for teachers to use specific experi- 
ments for assessing some parts of manipulative skill, parti- 
cularly yield, purity and accuracy. However, many teachers 
feel that a grade based on a general impression of work over 
a period of say a term or even a year is more appropriate for 
the less exact aspects of manipulative skill which can only be 
assessed by direct observation. If periodic grading is used, 
it is important that teachers should consciously look for 
evidence of th© skills throughout the period and not simply 
rely on their memory at the end of the period* (A further 
note on grading by impression is given on p* 74) 

5 . For the purpose of this assessment at ti tud e 3 can be 
divided into three categories: 

(i) Willingness to co-operate in the normal routine of a 
chemistry laboratory. This will include a knowledge of 
safety regulations and a wi llingness to comply with them and 
the other regulations which are necessary for the efficient 
running of a laboratory. 
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(ii) -Persi stance ami resourcefulness - the will to work 
u r iaid.&d on set problems and to see them through, to a nieaning- 
ful conclusion. 

viii) Enthusiasm: a commitment to practical work as a worth- 

while pursuit without compulsion. This could ow itself in 
suggestions for lines of investigation not specified in the 
course and in assuming the more active role when working in 
groups , 

A grading at the end of each year, or even at the end of 
the course, is probably the most appropriate method of assess- 
ing attitudes. 

Areas of subject matter 

The assessment in the future will not be restricted to 
specified experiments as it was in the trials. Teachers will 
be free to choose which experiments to use and these experi- 
ments need not form part of the published huffield course. But 
in order to get a reliable order of merit it is important that 
“fc © main b.t eas o£ subject rn&t-'tejr are oov6;pedL* On each student’s 

record card the examiners will expect to see a record of assess- 
ment on at least eight experiments, including at least one from 
each of the following areas of subject matter. 

1, Changes in substances and patterns in changes in 
substances. (This is a deliberately broad cate- 
gory to include most of the course not covered by 
2, 3 and 4 below. Most of the assessed experi- 
ments are likely to come in this category, ) 

2. Equilibria. 

3, Kinetics. 

4 . Energetics, 

Types of work 

Three main types of work can be identified in the course 
and the examiners will expect ‘ to see each one used in the 
assessment, but not necessarily with equal weighting. 

1. Quantitative work, 

2. Preparative work. 

3. Qualitative work. 

Allotment of marks 

The number of occasions on which marks are allotted for ' 
practical work is left to the discretion, of the teacher provided 
that the minimum number of experiments' is attained. . it is . 
desirable that the objectives, with the possible exception of 
objective 5> are assessed on more than one occasion in order to 
get a reliable estimate of ability. 

\*r. 
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Students should be told at the beginning' of the course 
that their practical work will be assessed and that the results 
will be sent to the examiners. They should also know, in 
general tens, the qualities for which the teacher will look. 

They should be told that the assessment will be more obvious on 
some occasions than on others, but that it will usually take 
place during normal work and not on special occasions. It may 
reassure them to know that the average mark for practical work 
will be higher than that for the written examination . 

The examiners will not necessarily expect that all the 
students in a teaching set will be assessed on the same expert* 
ments, but an equal balance of objectives is desirable* Assess- 
ment of objectives 1 and 2 will be difficult in experiments in 
which the students work in groups, but teachers should not feel 
restricted in the assessment to those experiments which are 
performed by students working alones some assessment of objec- 
tives 3, 4 and 3 is possible in group work. 

During the trials a grading scheme (3-4-3-2-1) was used to 
allocate marks to a particular objective on a particular occasion, 
and teachers may find a five point scale useful especially for 
those parts of the work which are marked by impression rather than 
by a mark scheme, but they are free to use other schemes if they 
wish . 

There are three main ways in which marks can be allocated 
and teachers will probably find that they have to use all of 
them . 

A mark scheme « This will be most useful when marking 

written evidence of observation, interpretation, planning and 
accuracy. 

2- Marking by impression on a single occasion . This will be 
useful for marking evidence which is less precise than that 
mentioned in 1. For example, a teacher may wish to assess 
dexterity in handling unfamiliar apparatus say in chromato- 
graphy. This can best be done by direct observation of the 
students at work, and in a large class it is not easy. 

Teachers should try to assess only one such quality by impres- 
sion during one experiment. Furthermore, teachers are usually 
too busy to do more than make a note of those who are doing 
well or badly; the rest can only be given the mark for average 
work . 

A good deal of thought during the trials was given to the 
question of whether "average 1 * should be the teacher f s estimate 
□f the average for his set or his estimate of the average for 
1 A* level work in general. The former has the disadvantage 
that a teacher cannot estimate the general level of practical 
ability of his set until he has been teaching them for some 
time,' The disadvantage of the latter is that it depends on 
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the teacher's experience of ' A * level work. Since the raw 
marks will be scaled to a norm anyway, it is their distribu- 
tion and the internal consistency of the marking which is 
important. But some uniformity is desirable and it has 
been decided that teachers should try to judge what is 
average from 'A* level work in general rather than from the 
work of their teaching set. Thuss 

Excellence of work unusual in an 'A' level student - 5 



Above average work - 4 
Average work - 3 
Below average work - 2 
Unusually poor work for an 'A' level student - 1 

5. Marking by impression after a period of time . This will 



apply mainly to attitudes to practical work, but teachers may 
wish to assess other objectives, particularly some of the less 
precise aspects of manipulative skill, periodically rather 
than in single experiments. The period is left to the teach- 
er's discretion ; it could be done once a term, once a year 
or, for some objectives, once at, the end of the course. The 

criteria for awarding a grade are those set out in 2. 

Information required by the examiners 

Three or four weeks before the written examination the 
examiners will require the following information from schools, 

1. All the record cards of the students arranged according 
to teaching sets, A sample showing how a record card can be 
completed is given in Appendix A; this should not be taken 
to be an example of an ideal assessment. 

2, A summary of the marks for practical work for each set, 
showing the total mark awarded to each candidate ae a percen- 
tage (Appendix B). 

5. Any other relevant information, for examples 

(i) Outstandingly good or bad performance throughout 
the course by individual candidates. 

(ii) Outstandingly good or bad performance by a set 
as a whole, 

(iii) Circumstances such as absence or change of school 
which could affect the performance of a candidate, 
particularly if it has resulted in the completion 
of fewer than eight experiments in which assess- 
ment has taken place. 

The moderation of marks * • 

The present need for moderation of the teachers ' marks 
has already been mentioned (p. 70), For various reasons, 
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including those of ineffectiveness and expense, moderation 
by visiting examiners , moderation by a practical examination 
and moderation by performance on the written papers as a 
whole, have been rejected* 

Moderation will take place by the performance of 
teaching sets as a whole in those questions in the written 
Papers I and II which will have been designed to test prac- 
tical work; these questions will form not less than 20% of 
the two Papers* It seems that performance in these questions 
ia more likely to reflect the general level of practical work 
of a teaching set than any other moderating instrument which 
can be conveniently applied. 

Three ranges of marks will be allocated, based on the 
performance of teaching sets as a whole in the written prac- 
tical questions* To conform with the findings of the trials 
(p* JO) all three ranges of marks will be narrow, they will 
overlap, and all three mean marks will be higher than the 
half mark. For example, in 1968 the maximum mark for the 
teacher assessment of practical work will be 70 , which com- 
prises 14% of the maximum mark for the whole examination and 
the three mark ranges will be: 

Top range ( 20 % of teaching sets), mean mark 57* 5 , 
standard deviation 4 * 

Middle range (60% of teaching sets), mean mark 47*5? 
standard deviation 4 * 

Bottom range ( 20% of teaching sets), mean mark 37*5 * 
standard deviation 4 * 

This form of moderation will be reviewed from year to 

year# 



In conclusion 

This is a first venture in this form of assessment of 
*A P level practical work in chemistry by teachers, and in 
many ways it is. still on trial. At first it will no doubt 

seem strange to both teachers and students, but it is hoped 
that in time it will become accepted as part of normal Sixth 
Form work* The examiners will welcome queries, comments 
and suggestions from the teachers who operate the scheme; 
they should be sent to Mr. J#C, Mathews, 51 Laverton Road, 
St, Annes , Lancashire* 
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Appendix A 

A SAMPLE OF A HKCOHD CARD SHOWING IMAGINARY MARKS FOR A STUDENT 



N.B. The reference numbers and titles of experiments are based 
on the written materials before publication. 



Narae School Form and set 

NUFFIELD FOUNDATION SCIENCE TEACHING PROJECT 
CHEMISTRY ADVANCED LEVEL 
ASSESSMENT OF PRACTICAL WORN 

Duration of courses September 19-.. to July 19... 
Record Card Number , „ # . 


ASSESSED EXPERIMENTS 
AND OTHER ASSESSMENTS 


OBJECTIVES (showing 1 number) 
AUD MARKS ( showing maximum ) 


2 . 2 o Proper ties of 

chlorides 


Observation ( 1 ) 
6/10 


Interpretation ( 2 ) 
4/5 


5* 3 (b) Iodine/i odate reac- 
tion by titration 


Accuracy ( 4 ) 

7/10 




b.l^a; Reactions of some 
metal elements 


Observation ( 1 ) 
8/10 


Interpretation ( 2 ) 
5/5 


7 t 2(b) Enthalpy of 
hydration 


Comment on sour- 
ces of error ( 5 ) 
5/5 • 




y»3(b) Preparation of 

phenyl benzoate 


Yield ( 4 ) 
2/5 


Purity ( 4 ) 
4/5 


General impression 
» oof manipulative 

skill (year end) 


Orderliness and 
methodical 
approach ( 4 ) 
4/5 


Speed ( 4 ) 
3/5 


*“■“ General impression 

of attitude to 
practical work 
(year end) 


Routine (5) 
4/5 


Persistence (5) 
4/5 



SUMMARY 




OBJECTIVES 






1 


2 


5 


4 


5 




Marks 












Signature of teacher 


Scaled 

marks 










... 



TOTAL 
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N s>jTi o •***»**»•**«•■*•■»** school Fo nji *«••>*•«■* 

NUFFIELD FOUNDATION SCIENCE TEACHING PROJECT 
CHEMISTRY ADVANCED LEVEL 

ASSESSMENT OF PRACTICAL WORK 

Duration of course* September 19.** to July 19* - * 

Record Card Number 


ASSESSED EXPERIMENTS 
AND OTHER ASSESSMENTS 


OBJECTIVES (showing number) 
AND MARKS (showing maximum) 


11.2(b) Determination of 
strength of 
hydrogen bond 


Planning (J) 
5/10 


Interpre- 
tation (2) 

3/5 




1J,3 Solubility pro- 

duct of silver 
bromide 


Accuracy (4) 

4/3 






13 . 6 (d) Prediction of 

redox reactions 


Prediction 

(3) 

3/5 


Observa- 
tion (l) 
6/10 




15 * 2 Rate of reaction 

tin and iodine 


Recognition 
of errors 

(3). 

4/5 






16,1 Reaction of thien- 

yl -chloride with . 

hydrates 


Observati- 
ons (l) 

4/5 


Interpre- 
tation (2) 

3/5 




17*3 Chromatographic 

separation of 
amino acids 


Dexterity 

(4) 

3/5 


Observa- 
tions (l) 
4/5 


Interpre- 
tation (2) 
2/5 


***- General impres- 

sion of manipu- 
lative skill 
(year end) 


Orderliness 

(4) 

4/5 


Speed ( 4 ) 
4/5 




-- General impres- 

sion of atti- 
tude to practi- 
cal work (year 
end) 


Routine 

(5) . 
4/5 


Enthusi- 
asm (5) 

4/5 





SUMMARY 



• iifitlciiali 

Signature 
of Teaoher 





OBJECTIVES 






I 2 


3 4 


5 


TOTAL 


Marks 


28/40 12/25 


15/25 32/45 


16/20 


IO 9 / 16 O 


Scaled 

marks 


18/25 7/15 


6/10 21/50 


16/20 


68/100 
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Appendix B 

A SAMPLE OF THE FORM QF THE SUMMARY OF ASSESSMENT 

FOR A TEACHING SET 



School Form and Set 

NUFFIELD FOUNDATION SCIENCE TEACHING PROJECT 
CHEMISTRY ADVANCED LEVEL 



SUMMARY OF ASSESSMENT OF PRACTICAL WORK 



Duration or Courses September 19*** to July 19< 



Candidate 

Number 



Name and 
ini tials 



Total Mark 

/1 00 




Leave 

space 



this 

blank 



Teacher's signature 



-Date , , , 



»*'*•# 
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III 



BACKGROUND PAPERS 



1 * Evaluation in Chemistry 

within a Second Level General Education Programme 



by J.P. AXles 



1 . 0 The main purpose of the paper and some preliminary comments 

This working paper is prepared for a conference relating 
to evaluation in chemistry within second level general education 
programmes * It is presented as a basis for discussion and 
study. 

The main purpose of the working paper is the presentation 
for discussion of the following aspects of the theme of the 
conferenoe # 

1.1 The learning and teaching of chemistry are on© component 
of a second level general education programme* The total 
second level general education programme will include, in 
addition to other science subjects, such as physios and 
biology, several other major components , such as first 
language, second language, mathematics and so on* Hence, 
"Evaluation in Chemistry" is one aspect of the larger 
problem of "Evaluation within Second Level General Educa- 
tion Programmes". Consideration and critical study of 
evaluation in chemistry, ■ to be complete, may demand a 
consideration of the general problems of evaluation 
within second level general education programmes * It 
may be appropriate to discuss whether all the issues in 
evaluation in chemistry are specific to chemistry, or 
whether many are only special cases in the fields of 
chemistry of a general class of problems of evaluation 

at second level in general education, 

1.2 In the design and implementation of educational pro- 
grammes many major processes are involved, such as the 
followings 

i. Analysis of the situation within the educational 
system as a whole, isolation of significant 
issues and professional policy decision making. 

ii. Planning and programming of educational activities. 
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iii. Communication of the planned programmes to execu- 
tions! levels along with relevant resources allo- 
cations « 

iv. Supervis-on , assistance and guidance in implemen- 
tation , 

v, Eval\x&tion and assessment in the context of the 
specified goals. 

It will "be seen that evaluation and assessment are hut 
one of the major processes taking place within an educational 
system. Henoe, evaluation in 'chemistry needs to he viewed 
from this "process standpoint" as well, and should be consid- 
ered in the perspective of the system as a whole. 

1.3 Evaluation in chemistry viewed from these two standpoints 
will be seen, to have many unresolved or partially resolved 
problems. It may be very important not only to isolate 
the fruitful and valuable techniques and approaches now 
available, but also to isolate and to recognise the major 
unr ©.solved problems which may require exploratory work in 
the immediate future. 

It is recognised that the participants at the conference 
are primarily concerned with the subject at the level of design, 
and the emphasis throughout is essentially on recognising 
useful concepts, techniques and criteria of judgment and 
decision that are available, and the recognition of new areas 
for action research. 

A working assumption in relation to the entire paper is 
that evaluation, as a process, is perhaps still in need of 
major development? and evaluation in chemistry in particular 
may see substantial progress in the years immediately ahead. 

It is relevant to remark that, to those who are not 
directly concerned with design and innovation and path-finding 
activities in the fields of evaluation, points 1.1, X»2 and 
1.3 above may have but little relevance. Some may even argue 
that not only are the viewpoints suggested of limited relevance, 
but also that the attempt to see evaluation in chemistry as 
suggested may be positively "confusing", This may be true. 

The paper will have relevance only in the context of a "design 
and innovation-oriented" viewpoint. 

In order to recognise “the status of evaluation in relation 
to classroom or school activities, questions such as the 
following need to be considered? 

What type of activity and teaching programme goes on 
within the school? 

Where does one get statements and descriptions 1 (speci- 
fications) of the school's programme of work? > ■ 
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What are the aims and purposes that the school seeks 
to achieve? In regard to these what significant 
problems exist? 

What staff are charged with recognising' and formu- 
lating these problems? Is there an active programme 
of work directed towards the solution of these prob- 
lems? 

Who decides what goes on in a school and who plans out 
in general terms 5 and later in detail 9 the programmes 
of work and study within the school? Who assist 
and guide principals,, teachers and pupils at work? 

Are there procedures and guidelines laid down for 
finding out whether the school is achieving the 
purposes that it has set itself to seek? 

How is the school associated with its immediate 
community? How is the school associated with the 
larger community* the nation? 

What methods and techniques and procedures are 
available for finding out whether the pupils are in 
fact developing adequately and learning effectively? 

What administration and management techniques have 
been established so that the work of the school is 
linked to the entire hierarchy of activities 
directed towards the development of the community 
and the nation? 

All these questions and many others of a similar nature 
are relevant in the context of a discussion on evaluation* 
Depending on the nature of the task in hand, some of them 
become more important than others. In this paper it is 
^ u S‘g' es 't e d that staff engaged in "Assessment and Evaluation" of 
pupils (and perhaps of the other aspects of the school system) 
should raise these questions explicitly and consider them at 
least briefly* 

One can get answer© to these questions which are specific 
but "static" in outlook. It is also possible to get answers 
that, while being specif io and operational, have an element of 
dynamism, fall into a meaningful perspective and, in particular, 
have the overtones of an "on-going" programme, adapting, 
assimilating, evolving and actively growing in many dimensions. 
It will b© accepted that the latter type of answer is more 
useful to the "designer" at all levels than the former. 

Hence, "design staff" in the field of "Assessment and 
Evaluation" may need to consider the scope and nature of an 
educational system as a whole » An education system - as all 
"systems" - can be viewed and analysed in many ways* "What 
particular pattern of analysis is specially relevant?" is an 
important question. 
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Usually design staff concerned with "Assessment and 
Evaluation 11 are also intimately associated with curriculum 
design and development* A discussion on "Evaluation and 
Assessment f * must, therefore, be based on a critical review 
of the general conceptual and methodological aspects of 
curriculum development. 

Curriculum development, viewed in the moat general 
terms, involves the following principal processes: 

Firstly, it involves a complex set of group decis- 
ions of a multi-disciplinary nature including many 
value judgments (the decisions being based on avail- 
able evidence, studies, expert opinion, body of 
theory and learning research), the total complex of 
decisions leading ultimately to a specification of 
objectives and content for a particular curriculum 
or educational programmes. 

Secondly, having regard to the availability of 
resources of men, materials and time, resources for 
learning and ins true tional procedure© are devised 
to set the conditions under which fruitful learning 
experiences are made available to pupils. This 
stage often involves work of an artistic, creative 
and synthetic nature. 

Thirdly, quality control and feed-back of evidence 
.needs to be designed and elaborated to ensure that 
curriculum plans are effectively realised. 

s Curriculum development involving the above processes is 
not a one-stage operation. The third process, indicated 
above, is essentially the process "Evaluation and Assessment”, 
the valid use of which transforms the entire education pro- 
gramme into s dynamic and continuous one* 

Evaluation and assessment are related to pupils, teachers, 
schools, communities and the whole system* It can also 
concern itself with the evaluation of simple outcomes in the 
field of knowledge and facts; or more complicated and more 
subtle learning outcomes relating to the application of 
knowledge , analysis , synthesis, eto* Furthermore , it can 
concern itself with the development of simple and complex 
skills at a variety of levels of versatility and adaptability! 
and it can also relate to the evaluation in pupils and 
teachers of the growth of appreciations , values and attitudes* 
In any given case one may be concerned with only a limited 
aspect of evaluation. But the designer and research student 
may need to see all these possibilities and these may need to 
be recognised in a coherent perspective. At ail times he 
needs to be conscious of what he leaves out of consideration! 
that which he leaves out he should leave out deliberately* 
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2 , 0 Evaluation in chemistry - one component of the evaluation 

of second level general education programme 

Second level education programmes, and in particular second 
level general education programmes, have been the subject of 
study and critical comment at national and international levels 
in the immediate past, A classic survey in this context is the 
"World Survey of Secondary Education" carried out by Unesco in 
1961. In this study and in related studies significant trends 
in second level education have been recognised. In addition 
to world-wide surveys , i-sgional studies are also available, and 
one that is directly concerned with the Asian region and relat- 
ing to second level education is "Perspective oi Educational 
Development - 1960-1980, Draft Asian Model", Unesco, Paris, 
1964 . Similar documentation and analysis are available for 
the African region and through OECD for Europe. The current 
educational scene is characterised by an unprecedented expan- 
sion of second level general education. Concurrently with 
this, many acute problems of articulation of various levels and 
types of education have arisen. In addition, major questions 
' have been raised as to the suitability of curriculum specifica- 
tions for this large pupil population with heterogeneous 
charae teristi eg , 

Furthermore, the rapid growth of science and technology 
have brought in major problems of educational design, especially 
as "pupil- time " is limited. The curriculum of general educa- 
tJ.cn needs to be harmonised with the widening concepts of human 
osf&ure . This aspect has been particularly underlined in a 

recent study by Unesco (Final Report - Meeting of Experts on, 
the Curriculum of General Education, Moscow, U.S.S.H., 1968 1 
Unesco, Paris), 

Contingent on the above changes which are taking place 
very rapidly indeed, significant changes are occurring in the 
methods and content of education as knowledge is accumulating 
at an accelerating pace and new ranges of skills and mechanical 
assistance are being offered to the teaching profession. 

In response to this and many other stimuli, educational 
designers in the last decade and those charged with the 
execution of educational programmes have sought to assimilate 
the new knowledge and techniques and adapt them to existing 
school programmes to meet the changing needs. How successful 
these efforts have been is a question that needs analysis. 
Curriculum development, curriculum implementation and curri- 
culum evaluation have become major activities within general 
educational programmes. Significant oonceptional and proce- 
dural guidelines have been recognised. (Vide Unesco . Report 
on Curriculum of General Education, Moscow, 1968* and also 
"Unesco Regional Programme for Promoting Educational Research 
in Asia - Curriculum Research in Asian Countries - Workshop 
Report", National Institute for Educational Research, Tokyo, 
1968 .) • • * 
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One of the mo 6 r. signif leant areas within the curriculum 
wherein active development work has taken place in the last 
decade is in the field of science education at the second level 
These development activities have occurred in the developed 
as well as developing countries* Curriculum development in 
chemistry and the associated programmes of evaluation in 
chemistry learning and teaching' is hut one activity within 
this extensive and v/orl d-wide ferment in second level education 

A programme of evaluation in chemistry must necessarily 
concern itself with objectives of chemistry teaching and learn- 
ing, And an analysis of the objective of chemistry learning 
and teaching will bs incomplete without an appropriate compari- 
son with general educational objectives associated with second 
level general education programmes. Statements at the general 
level and the statements in specific terms for chemistry had to 
be consistent even though the level of resolution and idiom of 
the latter . are different. A relevant question, therefore, is, 
what are the priority general objectives of second level 
general education programmes? In what direction are these 
objectives changing substantially or in terms of emphasis? It 
is not possible to review this issue comprehensively here. It 
may be sufficient to refer to a recent summary at the inter— 
national level . The relevant reference is given below. 

Meeting of Experts on Curriculum of General Education 
(Moscow, £)*S*S,R», 16-23 January 196Q) , Final Report 
ED/CS/ 4 /IX, Gnesco, Paris, June 1968 . 

A consideration of the above study as well as such studies 
as are reported in ~ 

Invitational Conference on Testing Problems, October 
1967 (Chairmans B.S, Bloom ) 5 Educational Testing 
Service, Princeton , /Mew Jersey, 1968* 

and 

A Selected and Annotated Bibliography of Studies 
concerning a Taxonomy of Educational Objectives - 
Cognitive Domain, R»G. Cox, R.J, Unke$ Learning 
Research and Development Centre, University of Pitts- 
burgh, 1967. 

would indicate that many problems of evaluation in chemistry 
can be profitably viewed as special cases of a general class 
or problems in relation to evaluation within second level 
education. i . ■ 

This is further reinforced by the viewpoints implicit in 
the Workshop Report - Curriculum Research in Asian Countries - 
N.I.E.R*, Tokyo, already referred to. 
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3 ■ 0 Evaluation in chemistry a major process in an eduoation 

system 

An educational system is a complex* highly organised 
entity. It involves thousands of schools* tans of thousands 
of teachers and non-teachers, and may involve several million 
pupils* All these individuals constitute a large organisation 
with explicit and implicit aims and purposes. 

Evaluation is one of the major processes occurring within 
this complex system. 

In recent years attempts have Lean made to study and 
analyse education systems using the concepts and techniques of 
"systems analysis' 1 . An attempt of a general nature in this 
direction is found in a recent study entitled* 

The World Educational Crisis - A Systems Analysis* 

Philip Coombs, International Institute of Educational 

Planning, Paris, 1968* 

The framework used in this analysis is general and is Indirectly 
relevant. The recognition of an educational system as a dynamic 
one with diverse "inputs" and "outputs" and responding to many 
"internal" and "external" pressures is a necessary aspect of the 
insight for staff at design and research level. 

Two modes of analysis having implicit in it a systems 
approach directly and indirectly relevant to "Evaluation" are 
given, in the Annexe to the study relating to the curriculum of 
general education cited above* (Report on. Curriculum of General 
Education - Moscow? Unesco, 1968*) 

Curriculum curriculum development and curriculum 

evaluation involve complex inodes of communication and execution 
within an education system* Among other things the adequacy 
of such communication will ensure that the system will adapt and 
grow* The dynamics of interaction that are brought into being 
when active curriculum development and curriculum evaluation 
programmes are initiated within an education system is very 
complex. The pattern and range of interaction vary from that, 
which occur at the level of policy decision makers (ministers 
and directional staff ) to the various complex interactions which 
occur at the level of principals, teachers and pupils within 
schools and classrooms* This latter group forme the primary 
focus of attention around which the entire education system 
operates. 

Curriculum development and curriculum ©valuation need to 
be designed and consciously directed recognizing the multi- 
plicity of these interactions and with the specific objectives 
of winning active participation and involvement of all the 
varying participant groups * 
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The recognition of the totality of these interactions, 
and thereafter recogni sing - evaluation in perspective within 
the system, may require techniques of analysis other than 
verbal if ones* Two schematic models which complement one 
another are given in the Annexe to the Final Report of the 
Moscow Conference on General Education. 



Detailed descriptions of these theoretical constructs 
are given in the Workshop Report of the iff. I. E.R., Tokyo, and 
in the Ministry of Education, Ceylon, pamphlet "Structural 
and Functional Aspects of an Educational System — Ceylon, 

1967”, ' * * * 



Xn the implementation of curriculum programmes many 
people are involved* These participants operate at a variety 
of levels. The participants fall into the following broad 
categories*? 

(1) One group of participants essentially operate at 
"survey" tasks and analyse the present curricula 
and teaching programmes, describing the present 
situation. An attempt is made to isolate signi- 
ficant problems associated with the system, 

(2) A second group of participants could be involved 
in assimilating information such as obtained above 
and recognising the problems specifically, breaking 
them into detailed problems of an operational 
nature and then trying to work out alternative 
lines of approach towards solution* 

(3) A third, group of participants could fee policy 
direction and design staff in education systems 
who seek to assimilate and adapt the information 
such as that obtained fey the above participants , 
and to plan out teaching programmes for the 
various education systems in which they function. 

These participants will fee essentially involved 
in designing plans and implementation programmes 
using selected preferred lines of approach* 

(4) Another group of participants would essentially 
fee charged with task© such as communicating the 
plans, with programmes of teaching, to executive 
levels within the educational systems, leading 
ultimately to communication and assistance to 
principals and teachers working in the schools. 



Although functionally isolated, the same individuals may thus 
often fee functioning in more than one group. 
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«> as a-.sasssis'a 

Bupervising the prop s . 

( 6 ) The last and most important group of 

K * could be teachers who will be seeing to implement 
the curriculura programme in the classroom. . 

g?oup is central to the .hole P*?^*“* ' VtJleli 
and other functions so as to assist th m 

work. 

»> .•Esa.nnsr-’sapff^S’'' 

irsi'KiS;" ™ 

ob j eeiivee • These ^riculum objectives and hence 

evaluation may relate to aspects of functione f 
design level staff, teaching level staff and 
assessment and evaluation of pupil growth ana 
development. ■ 

A. has been indicated, a curricula, development end^i.ple- 

mentation programme can fail to % J? 4 goals effectively, 

fail to operate adequately and ach ^ eve ^® S ^ e ^ h J se vari0 us 
on account of inadequacies at one or m r ,j nro- 

levels of participation. In short, general education pro 

grammes may fail because na^appropriate contacts 

adequately »ith proper consultation. and “ p ff° p r0 eiraie staff 

~ .hiSh_.ae.not oonprahensive , 

^qia^andihrp^^d tfn ^"r^ional 

consistent and workable, the message given t inadequate, 

level, in the ash ^t.ti» 

^s^rsrs:. ( Six «* 

tion of resouroes of men, -‘«^aii^ of^plo.e^t^n^ay not 
be^igh^beoaus .^supervision a?.d aeeLtance has not been given 
in full measure. When all of the aspect. ■>«»”£« ‘i,' ' 
then the entire situation can bo t ii a e) afta? ouestage 

greesive characteristics (tending ^ n&daouatt E^alus- 

If dav.lop.ent if evaluation programmes “• on _ going 

tion and assessment are not only ■ _ ranging all tha .ay 

programme, but must occur at all th _ * be directed 

to classroom implementation. Evaluatr^ ± nade auac i e 3 that have 
towards locating all the v ^ 10aa , ^f ^oLt ^ven would indicate 
been referred to above. The brief »°°®^ a gT“ in an 

the critical role that assessment an J .^^Wange was not very 

^WhTSr: f urgency r^lluvely low, thence need for 

illicit — - 
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rapid feed-back and sensitive response to evaluation data 
becomes a vic^l necessity in any programme of work. 

In a programme such as design and implementation of 
general education programmes where M single -valued real 1 ’ 
solutions cannot he worked out from established theoretical 
frameworks, there is a need to monitor continuously the 
validity and workability of design programmes at the imple- 
mentation level* Evaluation is not an activity that is limited 
to simple evaluation of pupil growth per se. It is true that 
ultimately the final test of the adequacy of an education system 
will be the assessment of pupil growth. If this occurs ade- 
quately and effectively , then to that extent it may be possible 
to argue that all other components are funotioning reasonably 
adequately. 

Usually, unqualified success of design attempts does not 
attend a complex implementation programme, such as in the field 
of general education* Only partial success is achieved, The 
limited success may be the result of limitations and inade- 
quacies occurring at all the various points in the implementation 
sequence outlined, above# Evaluation must be directed towards 
achieving an understanding of performance characteristics and 
also towards diagnosing the particular problems which could 
arise and cause the partial achievement at any given level. 

Curriculum development involving all the above complex of 
processes is not a one-stage operation. The continuous nature 
of curriculum development and implementation has to be recog- 
nised all the while by all the various participant staff 
indicated* 

Evaluation is an Integral part and process within th© 
larger programmes of curriculum development in an educational 
system. Well planned evaluation programmes can be an immense 
asset In curriculum implementation. 

Ultimately the success or failure of an education programme 
must necessarily h© assessed by the adequacy with which it die- 
charges the assignment given to it by the community it serves. 

The assessment of community growth itself must therefore besom© 
an integral part of the evaluation. The assessment of the 
pupils, teachers, principals, supervisors, adminis trators , etc,, 
may well be elements which will make up this final assessment 
and evaluation. 

The school system is essentially an agency established by 
the community and is the main agency through which a community 
develops its ideals and attitudes and culture through genera- 
tions - conserving it, developing it and extending it. Every 
moment, everyone is actively going through processes of 
education and change of one type or another. This education 
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may be imparted in an informal manner through the various mass 
media techniques or in a formal manner through the established 
institutions such as schools, etc. The focus of attention in 
the current paper is essentially on the role and function of 
evaluation within and related to the formal agencies. 

kt each of the various functioning levels within an educa- 
tional system, evaluation needs to be carried out in signifi- 
cant dimensions. An evaluation programme to be comprehensive, 
consistent and adequate in practice must concern itself with 
all the various possible dimensions, namely: 



the dimensions of cognitive achievement ; 

. . * 

the dimensions of peychomotor execution ; 

,3t 

the dimensions of affective involvement . 



ana 



The evaluation of any individual or group at any one of 
the levels within education must ideally he concerned with 
evaluating in all of these various dimensions. It may well 
that the particular problem in hand and the limitations of 
current conceptual and technical knowledge and skills may com- 
pel one to limit severely the aspects which will be subjected 
to evaluation. While this may be so it is necessary that 
design staff should recognise explicitly, and at each level, 
in each situation, the various aspects which their evaluation 
programmes leave out of consideration. It would be valid to 
argue that design staff have no justification for leaving 
things out by default, forgetfulness and lack of attention. 

On the other hand, if limitation in knowledge, skills, etc., 
compel one to leave things out then it would be appropriate 
to leave aspects out explicitly and deliberately, recognising 
that all such omissions bring in limitations in relation 00 the 
interpretative value and. usefulness of the data. 



* The details of these various dimensions and the sub-levels 
within them have been described in the context of curriculum 
development and evaluation in an associated wonting paper 
entitled "Theoretical Constructs Relevant in Curriculum Devel- 
opment and Evaluation". This study uses a conceptual analysis 
which is essentially an adaptation of the analysis of Bloom, 
Krathwohl et al. in this field. The scheme of analysis xn the 
reference cited is comprehensive, in that it includes the psycho 
motor domain as well. 



FILMED FROM BEST AVAILABLE COPY 



- Si - 



2 , Evaluation - a Critical Stimulus 

to Science Curriculum Development in Ceylon 

by J . R&tnaike 



Generalised patterns of project planning - and management, 
applicable to such diverse areas of human activity as indus- 
try, government, educational planning, defence and warfare, 
have appeared in the last few years# Regardless of the 
sophistication of the projects, their specific or general 
nature, or whether they be limited or extensive, the frame- 
work of essential, integrated, cyclic, unit operations 
(decision making, planning, communicating, executing, con* 
trolling, evaluation) has been found to be versatile and 
effective as a conceptual tool for programme design and 
management , 

Useful transfer of these ideas to the planning of 
science curriculum development has been made# The flow 
sheet of the operation elements of the Ceylon Curriculum 
Development Project (Figure l), since 1958, reflects the 
essential components and patterns of such generalised 
analyses . 

The purpose of this paper is to summarise the contribu- 
tion which one important element - evaluation - has made to 
the Ceylon Project, and to identify areas where further 
action-research is required. An underlying feature of the 
action-research programme involving evaluation, and in fact 
involving all the other elements of the project, is the extent 
of realistic compromise from ideal situations that was made, 
particularly because of resource constraints, in a developing 
country like Ceylon, engaged in a major curriculum develop- 
ment programme. Nevertheless it will also be recognised 
that even without brilliant and sophisticated break-throughs 
in ideas and techniques, if a comprehensive, consistent and 
workable plan is designed, a considerable amount of construc- 
tive and productive preliminary activity may be successfully 
completed - within the restrictive framework of limited time, 
and human, and material resources. 



Phase One 



Planning and decision making for a project requires data 
relevant to the situation. Some subjective evidence was 
available that the teaching in Ceylon was didactic and aca- 
demic, and coaching for examinations the prime objective of 
the "teaching’ 1 . The question naturally arose as to what the 
characteristics of these examinations were. This generated 
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FLOW CHART SHOWING MAJOR ELEMENTS 
OF THE CEYLQfl SCIENCE/MATHS PROJECT 

(from the SEAMUS Report on the Ceylon Project) 
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the need to define specifically what objectives were to be 
examined. Neither the Education Department, which bore the 
responsibility of teaching pupils for the SSG/QC.L *0* Examin- 
ation (pupil age l6+),nor the Examinations Department, which 
evaluated the pupils and issued them certificates, had 
specified objectives for the examination in significant 
dimensions sufficient to locate inadequacies in the exanina- 
tion and show guidelines for correction. 

The definition of objectives was indeed a major operation, 
for it meant not only the translation of socio-political 
objectives into objectives of science teaching s but also 
defining the objectives in such a manner as to provide opera- 
tional cues for the design and assessment of science teaching* 
The pregnant social changes that had taken place in Ceylon 
since 1956 brought in their wake major changes in the nature, 
compost ti on , hopes, aspirations and' functions of the school 
population. The percentage of pupils in the age range 14-16 
who had some opportunity of utilising their science learnings 
at higher levels was less than Of all the 14 ya&r old 

pupils in the science classes, 95?^ were to join the general 
public of Ceylon for whom this was a terminal course in science. 

The urgency of the task of curriculum development, and the 
limited number of trained personnel, prevented an extensive 
and intensive analysis of objectives in all their multiple 
dimensions, at the start of the Project. The action-research 
programme isolated some of the critical ones, in one dimension , 
and used these for the analysis of the examinations and as a 
preliminary ■working base for further work or classifying 
objectives and planning* 

The preliminary set of cognitive domain objectives used 
was as follows (each of these was further broken down into 
sub-objectives) i 

, I Ability to recall important information* 

II Ability to apply principles qualitatively. 

Ill Ability to apply principles quantitatively . 

IV Ability to use the methods of the scientist. 

It w s recognised that, at this preliminary stage, no 
attempt would be made to work out or utilise objectives for 
the affective and psychomotor domains, and at that time (1958) 
no analyses or taxonomies for these domains were available to 
the Project. 

The analysis was made in tabular f orni K and the results 
for the years 1952 to 1958 clearly indicated that factual 
recall was indeed the major objective being tested in each 
of the science subjects* The application of knowledge 
appeared in less than IGyb of the questions, and hot a single 
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question required the pupils to analyse an experiment and its 
results for validity, or design considerations, or assumptions, 
nor to synthesize an experiment for a given situation. o 

questions appeared in which a pupil had to evaluate a situa- 
tion in terms of internal criteria, or show his j.evel or 
achievement in using the nature of proof. The relative 
crudenesB of the objectives framework used did leave some 
room for argument as to the precise objective under which the 
questions were classified. Also here was an inherent ambi 
guity since it was always possible that what was classified 
"application" could be "factual recall", for it depended on 
the vaguely defined learning experiences of the pupils. 
Nevertheless, it was clear that these examinations require 
the pupil to exhibit his powers of memory much more than his 
level of understanding and critical thinking. It was no^ 
surprising that when examinations demanded essentially recs-i 
of fact, the teaching, geared to the examination, a^so 
encouraged regurgitation of fact* 

The second characteristic of the examination investigated 
was content coverage. This was done for the same years as 
above, and the content areas were defined by the Examination 
Syllabuses of the Science Subjects, issued by the Examinations 
Department. The aub-division of the content areas was based 
on the usual content units as they appeared m textbooks and 
which were traditionally used in classroom teaching, 
tabular form it was clearly visible that certain content areas 
in each science subject were sampled many more times than o 
(nitrogen in the chemistry examinations times, equilibrium 
5 times, while heat and chemical change not once, m 14 exami 
ations). Furthermore, with a pass score of and with 

optional questions present in the science exaains ion papers, 
a pass could be obtained by a pupil who had covered a minor 
portion of the content areas in a particular subject. Repe- 
tition of stereotyped questions from a relatively limited 
number of content areas enhanced the possibility of guessing 
examination questions, and encouraged pupils to leave out 
large areas of the syllabus. 

A further significant characteristic was investigated - 
the pupil performance in these examinations. In spite ol the 
paucity of content coverage, the fairly high predictability of 
the examination questions, and the demand for t 

recall of fact at the expense of higher levels of achievement, 
the number of failures at these examinations was high.^ 
those who failed, failed very badly. More than a third of th 
pupils who sat scored less than 2^/100* 

This analysis raised .the need to investigate the reasons 
for such failures and wastage, at least in terms of standards 
and facilities for teaching the curriculum and its relevance, 
its adaptation to the needs, interests and individual di^er- 
ences of the pupils, and its communication, the amount and 
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quantity of supervision and guidance available to teachers 
and pupils, promotion policies, and gocio-econoiBic factors* 

These relatively simple analyses, involving: very little 
training and expense, were able to provide a very useful 
though limited profile of the examinations and, through them, 
the kind of teaching that had prevailed up till then. They 
were also able to isolate several critical problems that 
needed immediate investigation and solution. 

Phase Two 



Accepting the fact that examinations exerted great power 
over the methods of classroom teaching, it was decided that a 
more valid examination should be held, to start with in chemis 
try, while still using the existing syllabus. The strategy 
was to accumulate experience over time, through a series of 
relatively small changes, and do so in a manner that ensured 
a built-in mechanism of improvement . 

This was done in 19&1, and was continued until the revised 
syllabuses were published* These examinations were speci- 
fically designed to correct the defects identified by the 
previous analyses* To obtain wider content coverage, although 
the same number of questions as previously was retained, each 
question was broken into several sub-questions or items* A 
detailed table of specification was used in the design of the 
question paper* Since the papers were of the free response 
type, to enhance the reliability of the marking, the questions 
in the paper were structured, so that answers had to be speci- 
fic and short* 

In addition, a refined, explicit and detailed marking 
scheme was produced in co-operation with Chief Examiners (who 
supervised the Marking Examiners) and the Marking Examiners. 

Each Marking Examiner filled in a special data form indi- 
cating the pupil performance of the first 40 papers he correct 
ed, and of the last 40 papers he marked* The selection of 
scripts was thereby assumed to be random and the sample was 
about 25 fo of those sitting for the examination. . The selec- 
tion was also assumed to be stratified in the following 
respects. At the start of the marking, the Marking Examiner 
would not be very familiar and conversant with the detailed 
marking scheme, but h© would not be bored, or working under 
the stress of prox±ia±y\ to the marking deadline* On the 
other hand, when he had almost completed his set of scripts, 
he could be bored and under stress but would be thoroughly 
familiar with the marking schemes. The pupil achievement 
profile of this sample was close to the profile of the total 
population sitting for the examination. 
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The data for the performance of each part of each question 
were collected and analysed for the following oharaoterietice, 
for all three media, Sinhala, Tamil and English! 

(i) Frequency of choice per question and per item. 

(ii) Frequency of zero marks per item. 

(iii) Difficulty indices per question and per item, 

(ir) Ordinary pass in whole paper vs choice of item. 

(v) Credit pass in whole paper vs choice of item. 

(vi) Ordinary pass in whole paper vs 5 Q f° score or more 
in item. 

(vii) Credit pass in whole paper vs 5° f* score or more 
in item. 

(viii) The distribution across the score range, 

((lv) to (iii) were indices of discrimination.) This analysis 
provided a more detailed profile than before. 

Equally important was the fact that all the mechanical 
aspects of the analyses could be handled by staff with 
hardly any specialised training or knowledge of statistics. 

Nor was specialised equipment required. In fact the entire 
operation for the three years 1961 to 1963 was done without 
even calculating machines, and with only the assistance of 
teachers and pupils at the school where trial teaching of the 
new curriculum was being conducted. Yet it was possible o 
obtain a variety and detail of data useful as feed— bac or 
curriculum development. These operations served as critical 
explorations in the techniques in evaluation to be used later, 
after the new examinations and the new syllabuses had been 
introduced in 1963* They were also effective for the in- 
service training for staff in a real rather than a contrived 
action-research situation. 

Phase Three 

Having gathered some experience in the use of tables of 
specifications, and in the preliminary analyses of data, 
in 1965 the new examination syllabuses were produced, together 
with prototype question papers. The tables of specifications 
utilised Bloom’s Classification of the Cognitive Domain. For 
the first time in a national science examination, an outline 
of the objectives (other than content to be tested) was also 
included in the syllabus . 

Each science examination new consists of a multiple choice 
(objective) type paper, and a free response (essay) type 
paper, with a relative marks weightage of 2*3* Also an 
extensive and detailed programme for further action-research 

10 z 
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in evaluation har 5 been, drawn up, and used for the new examina- 
tions since 196b. By 1965 a number of personnel had already 
been trained, and superior facilities, such as computer-time 
and calculating machines, were available. 
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The study of the examinations is being done on a 
national sample, stratified in terms of educational regions, 
type of school, medium, sex. The kinds of analyses involve 
performance correlations among the objective type papers and 
the essay type papers within each science subject, and across 
the science subjects, correlations made across first language, 
mathematics and second language, for validity studies; 
difficulty and discrimination indices for item analysis; and 
effects on pupil performance due to factors sxich as sex of 
pupils, geographical region, type of school, urban/rural, 
teacher experience and qualification, parental education occu- 
pation and income. As a basis for investigating predictive 
validity of the examination, a follow-up study of. each sample 
will be carried through for ten years. 

The production of a general aptitude test has started, 
and trial testing will begin in a month's time. This will 
be used later as an additional dimension in the validity 
studies , 

While the investigations mentioned above were of prob- 
lems having direct relevance in the present stage of develop- 
ment of the theory and practice of measurement and evaluation, 
and capable of being wholly or partially solved on a short- 
term basis, the Project was not unmindful of problems which, 
though significant, were of such a nature that they required 
long- term fundamental research and a significant break-throug 
in the conceptual viewpoint for their solution. The working 
paper's produced on the restructuring of the Bloom classification, 
and the outline analysis of the psychomotor domain are an 
example of this, ' Other problems are those which arise as a 
result of the assumptions, limitations and restrictions that 
apply to widely used measures, theories and methods of educa- 
tion measurement and interpretation, euoh as the nature and 
characteristics of s 

the process of measurement , 

the process being measured * 

the instruments used for measurement, 

the interaction between ohe or more of the above. 

The iimitationa ■ . ' ' v ' 

The cyclic operations (decision making, planning, commu- 
nicating, executing, controlling, evaluating, ideally take 
place at each level of policy making and/or execution. In 

A 'j*'. i-. 
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the field of science curriculum development, one level at which 
these cyclic operations take place may correspond to the commu- 
nity and its functional organs, such as Parliament and the 
Ministry of Education. 

The next level may correspond to the Directorate of the 
Ministry of Education} the third level to field and supervisory 
staff; 'the fourth to the principals of schools} the fifth to 
the teacher, and the sixth to the pupil. Each level would 
contain the ' operations of decision making, planning, communica- 
ting, executing, controlling and evaluating, and each level 
would be articulated with the next through the operation of 
comsiuni cation* 

In this context, for the teacher and pupil levels, the 
widely used triangular pattern (Figure 2) may be thought of as 
a simplified and reduced model of the general structure (Figure 
5) , where decisions in relation to content of teaching, plan- 
ning of the organisation of this content, the mechanics of 
communication and the controlling of the pupil are coalesced 
into a general activity described aa ’'Teaching". . Evaluation 
is considered, apart. This . particular reduced pattern is ye p 
ful in focussing attention, on evaluation. (In considering 
contexts other than evaluation, where the focus of attention 
may be another aspect of the teacher-pupil situation, another 
reduced form may be more .appropriate. The reduced form also 
depicts the continuously changing and repetitive role of the 
teacher in the classroom, as one who operates to facilitate 
learning , as one who assesses pupils’’ growth.) 




Figure 3 
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Figure 2 
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At each of these respective levels of execution, and for 
each unit operation, within each level, the process of evalua- 
tion has to be considered aa an inseparable component and an 
essential means of feed-back and self-correctj.on e 

While the problems of pupil achievement evaluation have 
been investigated fairly extensively, at least ^ ! n ?°Snils 
domain, because examinations and examination results oi pup 
have an immediate and tangible socio-economic importance and 
high powers of motivation, and even though their contribution 
by way of feed-back of pupil performance and specification o£ 
the test items is very significant and effective, it must be 
remembered that measurement of the change in behaviour 
16 year old pupils is not the end product of this series of 
operations which started with the conversion and translation 
of socio-political objectives into teaching objec ives 
filtered through the various levels of execution.'. 

The end product is the type of behaviour exhibited by the 
pupils after they leave school and become citizens °f™® 
country, whatever their fields of endeavour may be, as buddhist 
monks or teachers, betel sellers or bus conductors, doctors or 
poll ti ons , lawyers or housewives, engineers or * of 

pupil's achievement at 16+ is not necessarily ai indica- 

such an end product, and it is certainly not a 
tor Ideally the process of end product evaluation shoul 
involve the evaluation of the magnitude and direction of change 
in the community. Even if it is temporarily assumed (however 
incorrectly) that pupil growth and achievement at lfa+ may 
taken as the -end product, the indices of achievement and growth 

of pupils (as represented by examination results are once, _ 

twice or thrice removed from many of the major elements _ 
programme, such as decision making, communication, controlling. 
Thefe indices are products of highly complex processes 
interaction, both positive and negative, among all the opera- 
S oil ll the levels. To evaluate these «^ XO&8 

by extrapolation from examination results of signals 

indeed a hazardoux process, and one unlikely n P. 
of sufficient specificity and discrimination to be used for 
improvement of the major operations • 

The Ceylon Project recognises this very serious 
arid while exploiting the examination achievement data of pupils 
“ ’the hae S i 0 i«ated action-research P«<trarmes to 

develop finally means of direct evaluation of * dl J h ® 
of science curriculum development* at all levels 
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Improvement of Evaluati on Practices 
A study of the G.C.E. *0 * solence examination 

by P. Samaranayaka 



The G.C.E. *0’ examination in physics, chemistry and 
biology based on the revised syllabuses was first Held in 
December 1965. The study of the examination "‘ W 1 ' 
performance was done on a national stratified random P 
selected prior to the examination, 

Naturo of the aiunglfe 

Of the pupils appearing for the examination ^° se 

amearinff for the first time from government and director 
managed fohoole and offering all three science subjects 
(phyfics, chemistry and biology) were considered for the 

sample. 

The .emtio strength wee about 10?4 of the above population 
and was selected in terme of the following strata. 

(i) Educational region, 

(ii) Medium of instruction. 

(iii) Grade of school'. 

In Ceylon there are fifteen educational regions broken 
down in terms of their geographical location. There are two . 
media of instructions Sinhala and Tamil, For the purpose o 
this study the schools were classified into two groups s 
schools terminating at the G.C.E. sr >» science level and e 
G.C.E. , A* science .‘level. 

The population was broken down to clusters where a clus- 
ter represented a school unit. 



Operations in the sample selection 



On the basis of information received from the schools 
regarding their G.C.E. »0» entries for ^.December ^65 exam- 
ination , a list of schools containing the following particulars 
was prepared* 

(i) Name of school. 

(ii) Educational region to which the school belonged, 

(iii) Media of instruction of the G.C.E. »0* science streams. 

(iv) Type of school (whether G.C.E. '0* terminal or G.C.E. A 
terminal)# 
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(v) Type of local body to which the school belonged (whether 
Municipal Council (MO) , Urban Council (OC), Town Council 
(TC), Village Committee (VO)). 

(vi) Humber of pupils entered for the examination in each of 
the media, Sinhala and Tamil* 

(yii) Pupil enrolment in the school from Grade VI upwards. 

Subject to the stratification referred to earlier, three 
different samples were drawn at random. These three samples 
were examined by a panel consisting of the Deputy Director 
General (Secondary Education), the Chief education Officer 
(Science) and another Senior Chief Education Officer with wide 
experience in the field. Their judgment was sought to sel ct 
the sample which was most representative in terms of an urban/ 
rural variable (whether pupil's home was in a rural area) 
which was not a strata factor considered m the selection of 
the three samples. The urban/rural background was used as 
crude index to reflect the socio-economic background of the 
pupil# 

On this basis 42 schools (clusters), of which 28 presented 
candidates in the Sinhalese medium while the other 14 in t e 
Tamil medium, were selected. The total number of pupils in 
the Sinhala medium amounted to 688 and in the Tamil medium 595, 
making a total of 1081 pupils. The sample was called the 
National Stratified Random Sample. 

Procedures for correction of scripts of the sample 

A specially selected panel of examiners^ were entrusted with 
the correction of scripts in physios, chemistry and biology of 
this sample. The panel consisting of 14 examiners per subject 
worked under the direct supervision of the Controlling Chi x 
Examiner of the respective subject, for a period of ten day.s. 

In addition to the correction of scripts, these examiners were 
called upon to fill in a sheet (Data Sheet I) in duplicate in 
respect of each of the pupils in the sample. The pupil res- 
ponse for each of the 40 questions in paper I .^respective of 
whether the response was correct or incorrect) was entered in 
Data Sheet I. In addition the score per question of paper u 
was also entered in this Data Sheet I. 

Finally a summary of the data in Data Sheet X was 
This summary gave the following particulars in respect of each 
clue ter. Th? nuubcr of pupil, selecting each choice I_ 
question in paper I, the number of pupils who omitted a parti 
cular question in paper I and paper II, the number of P*P lls 
who failed to score in each of the questions in paper II. Data 
Sheet I, in duplicate, had to be completed by these examiners 
under the direct supervision of the Controlling Chief Examiner 
of the respective subject. 
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Mode of analysis of Bata Sheet I 

Data Sheets I were scrutinised across subject, and any 
pupil who had absented himself from any of the subjects 
physics, chemistry or biology was struck off from the sample. 

The number struck off from the sample on this basis amounted 
to 29, which was less than 5f° of the sample. The final 
sample had a total population of 1052 pupils. 

Data Sheet I was used to process the following information. 

(i) Difficulty index of each of the questions of paper I, in 
respect of each medium and total sample* 

(ii ) Response per cent in respect of each of the choices of 
the questions of paper X in respect of each medium and 
total sample, 

(iii) Percentage of omissions in respect of each of the questions 
of paper X in respect of each medium and total sample. 

(iv) Discrimination coefficients of each of the questions of 
paper I . 

Other data 

A file is maintained in respect of each of the pupils in 
the sample which contains his G.O.E, >0' answer scripts of 
physics, chemistry and biology. In addition a copy of his 
cumulative record is maintained in his file. His performance 
through achOOl and any other ©duostioaal institute- ie ale-a kept 
track of and these particulars are also recorded. 

In March 1966 the Vedanayagam Non-Verbal Intelligence Test 
and the Jayasooriya Verbal Intelligence Test were administered 
to these sample pupils and these work sheets are also kept in 
their respective files. 

In addition to Data Sheet I, another sheet for recording 
pupil data was designed. This sheet, referred to as Data Sheet 
II, contains the marks of the pupils in the sample in the 
following papers, 

(i) First language 

(ii) Second language 

(iii) Mathematics / arithmetic 

(iv) Physics 

(v) Chemistry 

(vi) Biology 

(vii) Vedanayagam Non-Verbal Intelligence Teat 
(viii) Jayasooriya Verbal Intelligence Test 
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These narks are recorded in Data Sheet II in terms of 
clusters. In addition to these marks, Data Sheet II records 
the teacher qualification and experience in respect of physics, 
chemistry and Biology. 

Teacher classification is in terms of five broad groups; 

(i) Graduate with over 5 years' teaching experience. 

(ii) Trained with over 5 years' teaching experience after 
training, 

(iii) Graduate with less than 5 years* teaching experience. 

(iv) Trained with less than 5 years' teaching experience after 
training, 

(v) Others. 

A record of the pupil's socio-economic status is also main- 
tained in Data Sheet II. These are maintained in respect ofi 

(i) parental income; 

(ii) parental occupation; 

(iii) parental education; 

(iv) type of local body in which the pupil has been residing 
for the longest period of time. 

Mode of analysis of Data Sheet XI 

Analysis of Data Sheet II is classified into two main groups 

(i) Calculation of strata population, mean, median and standard 
deviation of physics I, physics II, chemistry I, chemistry 
II, biology I, biology II, first language, second language 
and mathematics/arithmetic in terms of each of the strata 
on which the sample was selected. 

(ii) Calculation of the inter-correlation among papers and 
different combinations of papers for the total sample. 

Analysis of other data 

Data in terms of each pupil's progress through a minimum of 
5 years after sitting the G.C.E. 'O' examination is in the pro- 
cess of being collected. Analysis and interpretation of this 
data has just commenced. 

1966 G.Q.E. 'O ' 

The nature of the population for the 1966 G.C.E. 'O' exam- 
ination was similar to that of 1965 . However, the size of the 
sample was brought down to about Sf>. This step was taken with 
a view to limiting the sample, population to ^andlabie numbers 
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as it was predicted that with time the population would increase 
gignif ioantly . 

The nature of the sample for 1966 was slightly modified from 
that of 1965 in that sex was also introduced as a stratum in 
addition to educational region, medium of instruction and the 
grade of school* 

The breakdown of the population into smaller waits 
where a cluster represented a school unit was founa , , 

unsatisfactory as certain schools had as many as two hundr 
candidates who were eligible for selection to the sample. If 
this type of cluster was selected for the sample it was evident 
that the sample would be heavily loaded in one direc ion or 
However, to eliminate such a school on these grounds would neces- 
sarily make the sample less representative. As «■ 
solution it was decided to break down each school unit 
simpler clusters where the maximum pupil strength P 6 ?' 
wasfixed at JO, On this basis a school which has 100 candi- 
dates eligible for selection to the sample is broken down to 4 
clusters where each of the first 5 clusters has 30 pupils and 
the last carries the balance of 10 pupils. 

Mode of selection of the sample 

As opposed to the 1965 procedure where the basic information 
regarding G.C.E. *0* entires were collected from the schools, 

1966 this information was collected directly from the G.C.L, Q 
entry lists of the Department of Examinations . 

In this manner the following information was recorded , 

(i) Educational region. 

(ii) Name ; of school. 

(iii) Grade of school. 

(iv) Type of management of school. 

(v) Medium of instruction. 

(vi) Number of candidates in each sex. 

(vii) Centre number. 

(viii) Inclusive index numbers of the candidates. 

The information collected was condensed and th 

educational region wise. With the help of this **J™* X «* 
total strength of the sample and the strength per 

determined. Having numbered the clusters m the ■« ® rder » 
five different samples of clusters were selected at 
subject to the above stratification using a table random 
numbers. Prom these five samples one sample was selected by 
a panel of three members consisting of the Deputy Direc 
of Education and two others so that this sample would reflect a 
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balance in terms of the socio-economic factors that operate in 
the population. This selection was done on similar criteria 
as used for the final sample selection of 1965. As the Sinhala 
population in the samples amou. ed to approximately 13/= of the 
population, only every other candidate of the Sinhala sample was 
included, while the Tamil sample was taken in tote. On thxa 
basis the maximum strength per cluster of the Sinhala medium 
population was brought down to 13 while that of the Tamil medium 
3 ion was JO* 



The sample for 196 6 consisted of 90 clusters representing 
88 schools and the total pupil strength was 802. Of these, 73 
clusters consisting of 552 pupils were in the hinhala population 
sample and 17 clusters consisting of 250 pupils were m the 
Tamil medium population. 

Panel of examiners 

The panel of examiners for correction of scripts consisted 
of 18 for each of the subjects physics, chemistry and biology . 

As in 1965, they were entrusted with both marking of scripts as 
well as entering of Data Sheets I under the supervision of the 
Controlling Chief Examiners of the respective subjects. 



Other data 

The collection of other data of the sample was 
manner similar to that of 



dona in a 



Mode of 



analysis^ 



of data 



similar to that 
.on and 10 



of 1965- 
pupils from 



Analysis of Data Sheets X was 
Nineteen pupils from the Sinhala _ _ 

the Tamil population had to be deleted from the sample, as 
pupils had absented .'themselves from one or more of the papers 
in physics, chemistry and biology. The sample was reduced 
773 pupils as a result of these deletions. Difficulty indices, 

- discrimination coefficients were worked 
in paper I of physics, chemistry 



response per 
out for each 
biology. 



cent and 
of the items 



to 



and 



slightly modified from 



Vedsnayagam M on-Verbal 
Verbal ■ Intelligence Test was 



The analysis of Data Sheet H was - 
the procedure adopted in 19&5. Analysis of scores of the 

- Intelligence Test and the Jayasooriya 
not attempted, as this G.G.B, 
population was outside the range for which these tests had 
originally been designed. Analysis of scores 
sample as well as per stratum was attempted in 
following papers: 

(i) Physics I 
(ii) Physics XX 
(iii ) Chemistry I 



•O' 



of the total 
terms of the 
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(iv) Chemistry II 

(v) Biology I 

(vi) Biology II 

(vii) Specific combinations of the above six papers 
( viii ) First language 
(ix) Second language 
(x) Mathematics 



In this analysis the population strength f mean, median and the 
standard deviation were determined in terms of each of the strata 

In addition, coefficients of correlation among papers and 
of certain combinations of p&pei’s were attempted* 

Other data 



The performance of this sample, too, is to be followed 
through the next few years for ultimate analysis. All the pupil 
in the sample are kept track of and a file is maintained in 
respect of each of these pupils* 

1967 Q.C,E, *0 f 

The nature of the population for the 1967 G*C,E, *0 r exam- 
ination was similar to that of 1965 and 1966 * The size of the 
sample was brought down further to 5 c /°* 

The nature of the sample for 19&7 was similar to that of 
1966 and the same strata factors were considered for the selec- 
tion of the sample. The breakdown of the population into 
clusters was slightly modified in that the maximum strength of . 
a cluster was fixed at 15# This was done by breaking down the 
school unit to simpler units of marianim strength 30 and then 
having selected these units of maximum strength 50 for the 
sample at random, ©vjbry other candidate in these units was 
considered for the clusters of the final sample. 

Mode of selection of the sample 

The selection of the sample was again done on the basis of 
information collected from the entry lists of the department of 
Examinations * On the above basis 5 different samples of clus- 
ters were selected at random and a panel of three members inclu- 
sive of the Deputy Director General of Education selected one of 
these so that the sample would reflect a balance in term© of the 
socio-economic factors that may operate in the sample. 

The sample for 1967 consisted of 100 clusters representing 
97 schools and the total sample strength was 779* Of these, J2 
clusters consisting of 569 were in the Sinhala ^medium population 
and 28 clusters consisting of 190 were in the Tamil medium popu- 
lation. 



FILMED FROM BEST AVAILABLE COPY 



- 107 - 



Panel of examiners 

The panel of examiners for correction of scripts consisted 
of 20 for* each of the subjects physics, chemistry and biology. 

As in 19b5 and 1966 they were entrusted with both marking of 
scripts as well as entering of Data Sheet I under the super- 
vision of the Controlling Chief Examiners of the respective • 
subjects. 

Other data 

• The collection of Other data pertaining to the sample is 
currently being carried out. The principals of the respec ve 

schools in the 1967 sample have been 0 ontao J ed r a ^^ e ^J“®® s O 
the pupils selected for the sample from each of their schools 
have been communicated to them. The required particulars of 
these pupils are now coming in. 

Mode of analysis of data 

Analysis of Data Sheet I was similar to that of 1965 and 
1966. 19 pupils from the Sinhala medium population and 9 Pupils 

fL the Tamil medium population were deleted from the sample on 
criteria similar to those adopted m 196? and 19 - • p 

was reduced to 755 pupils as a result of these deletions. 

Difficulty indices, response per cent and discrimination 
coefficients were worked out for each of the items m - aper 
physics, chemistry and biology. 

For the 196? sample only this data has been analysed so far. 
Analysis of Data Sheet II has not been attempted as the collectio 
of ’the data has not yet been completed. However, an analysis 
similar to that of 1966 is envisaged for Data Sheet 11. 
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Interpretation and use of data 

The G.C.E. r 0 1 examination data for 1965 and 1966 have been 
analysed to an appreciable extent and some tentative interpretations 
of this processed data have been attempted. 

One of the major changes in the examination design • ^ka£le 
effected in 1965 was the introduction of an < obj ec 

fixed response question paper for each o in ® ^ questions (Paper I) 

Lotion paper (Paper 

II). Paper II had the characteristic of the eeeey-type dues 
papers of the 1961-1964 G.C.B. 'O’ chemistry examinations. 

At this juncture it is desirable to investigate whether Paper 

Lis ilftr tfexam^e ‘ho^Lr Pa^"l ' s ' of the science subjects 

113 



FILMED FROM BEST AVAILABLE COPY 



- 108 - 



correlate with the Paper H'e. The correlations for Fhyeice I 
+ Chemistry I ♦ Biology 1 ts Physios il * 0h«: I.' try + 

QS y II for 1S65 and 1966 are of the order of + 0,9. corre 

SiL of Paper 1 to II for physics, chemietry and biology 
taken individually are also of the order of + O.b and + U.l for 
1965 and 1966 respectively. This order of correlations may 
interpreted as significantly high. It means that the n^w 
design of the examination has helped xn the improvement 
marking reliability and content and objective validity, whil 
essentially testing similar factors. 

The absence of suitable aptitude tests and I.Q. tests for 
the Ceylon 0.0.8. 'O' level child and the absence of horizontal 

projections usable for predictive purposes is unfortunate. 

However , the criterion of what is traditionally called . good 

student, i.e. performance not only xn a f ien ®! rd ^ for 
in first language and mathematics, may be used as an index tor 
judging addenda papers in the examination are behav- 

ing significantly differently to the other papers of the C.C h. 
•O' examination. This may be achieved by examining the corr 
lation coefficients between the performance in all s n) 

papers (i.e. Physics I & II, Chemistry I & II * ' 

with first language and mathematics. xhese are of 1 
5“ 0.7 for mathematics and + 0.4 for first language for both 

1965 and 1966* 

Some tentative trends in achievement are becoming visible 
in terms of the urban/rural background of the child, 
economic and educational status of the parent. But * ar “ ' 

data' is required in support of these trends before they may be 
presented for discussion. Similarly predictive validities of 
the examination would require several more years whx ^ . 

df the stratified random sample occupy positions at t . _ 

si ties, in other higher educational institutes and in th p 
fessions. ; 

The data has in part been used for the design and modera- 
tion of the science papers at the G.C.E. 'O' examinations o 
1966 and 1967. The data carrying an item analysis of the 
previous examination is made available to the Contro^ g 
Examiners and the moderators of physics che mist: ry « ^ 

at the time of setting and moderating the G.C.E. 0 papers. 

Some of this data has also been used to isolate specific 
areas of difficulty for designing in-service programmes. 

At this stage no attempt is made to make a®*®** 1 ™ 8 * 116 
way or another. However, with time and further data it i 
proposed to report other significant trends ^ problems on^the 
teaching and learning of science m Ceylon at the O.Q.E. 
level.. 
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4 # No tea on Structural and Functional Aspects 

of an Education System 

(Extract from pp. 2J-28 
of Ministry of Education Ceylon publication) 



The Ministry of Education has in the recent past initiated 
and carried out a programme of work relating to the General 
Certificate of Education, Ordinary Level, science subjects. 

In the years immediately preceding i960 several aspects related 
to the science teaching situation at the General Certificate of 
Education, Ordinary Level, were carefully examined and analysed. 

In particular, the specifications for teaching relating to the 
General Certificate of Education, Ordinary Level, physics, chem- 
istry and biology were considered# The_ examination syllabuses 
at the General Certificate of Education, Ordinary Level, for 
these subjects were critically analysed. Specific problems 
and limitations were recognised and decisions were made to 
pursue development programmes directed towards the improvement 
of the specifications for teaching and the specifications for 
examination# 

In making these decisions, relevant data from within the 
educational system were used. In addition, relevant data from 
other educational systems elsewhere in the world were examined 
and considered. 

In term© of the decision to design improved specifications 
for science teaching at the General Certificate of Education 
Ordinary Level, senior experienced teachers from within the 
Ceylon educational system, and outstanding consultant from 
outside , were invited. A team of workers numbering approxim- 
ately 10 to 20 worked on the programme continuously * At 
intervals of time, groups, as large a© 90 to 100, gathered 
together, for short periods, to participate in specific aspects 
relating to design and planning. Professional viewpoints such 
as are available with scientists, university science teaching 
staff, and research organisations , such as the Ceylon Associa- 
tion for the Advancement of Science, were channelled to the pro- 
gramme to make the plan adequate* Th® extent of participation 
was limited by time and funds. 

The communications so designed - in this particular instance 
"Syllabuses of Instruction and Schemes of Work" and the Teachers 1 
Guide Notes - were not directly issued to the field by the 
supervisory staff until some executive aation had been taken. 

Some of these executive operations related to simple 
routine matters such as writing the syllabuses of' instruction 
and schemes, editing and printing them in sufficient numbers. 

It was also necessary to set afoot relevant machinery for the 
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storage and distribution of such material. It will be recog- 
nised that thousands of copies of schemes of work and sylla- 
buses of instruction for issue to over 5GQ schools needed 
specific organisational and administrative action. 

A major preliminary executive action was the organising 
and running of pre-trial teaching of the material in a few 
selected schools for checking out the time base and other 
such aspects. 

There were also such matters as the design of suitable 
standard minimus* lists of equipment and the ordering of the 
new materials necessary for teaching the scheme. Th© recruit- 
ment of additional supervisory staff and 'their training to 
handle the new schemes were also necessary. 

Having pursued these relevant executive matters , and 
others not mentioned above, the plans and specifications for 
the teaching ( General' Certificate of Education, Ordinary Level) 
of physics, chemistry and biology were communicated to the 
teachers, principals and other field staff. This entailed a 
particularly complicated programme of communication to over 500 
schools located in many educational districts. Though the 
syllabuses of instruction contained relatively detailed "Fore- 
words" and "Prefaces” relating to the programme, it was never- 
theless necessary to arrange for personal meetings of the 
design staff with the field staff. This was done in all th© 
districts to a limited extent depending on the availability of 
time and other resources. 

The pupils* performances in the field at the General 
Certificate of Education, Ordinary Level, examination were 
carefully analysed in all the subject areas, using a care- 
fully stratified random sample of pupils selected from the 
examination enrolment. Th© performance of this sample of 
pupils was subject to detailed study and analysis* 

The whole programme of decision making, planning, communi- 
cating, supervision and evaluation is one that proceeds cycli- 
cally and unendingly. Therefore as a logical consequence of 
this recognition there should be continuing action-research 
programme directed towards assessing pupils 1 performances at 
the General Certificate of Education, Ordinary Level, in the 
relevant subject areas annually* This provision and decision 
will lead to the availability of feed-back information. In 
the light of such feed-back information, the total programme 
could be kept under continuous review and new decisions could 
be made and new programmes of planning and reconstruction could 
be set afoot so that the up-graded and the up-dated pattern of 
teaching and learning would be maintained at a high level of 
adequacy through time. 
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The principal of a school who had received the cossniuni - 
cation that science was to be taught in the General Certifi- 
cate of Education classes on the basis of modified syllabuses 
of ins fc.ru.o t i on and on the basis of amended, syllabuses of 
examination needed to ask a number of questions* Some of 
them were as follows i 

How much time was to be given to the subject? 

How were these periods to be organised in the day and 
in the we ex? 

What problems were there in organising the periods for 
work in the laboratories? 

How many double periods needed to be provided? 

Where should the theory periods be taught? 

Which teachers should be commissioned to do the 
modified syllabuses? 

Is* the equipment in the school sufficient for doing the 
modified programmes ? 

*.*.*..*., a no a series of similar questions* 

Having analysed this situation in the school and recog- 
nised the major problems in collaborative discussion with ih© 
teachers he would ideally plan out a programme of work in 
general; design the timetable, in particular; and attend to 
such other matters of planning a.s were necessary. 

Once this stage had been carried out, the teacher would 
be instructed to teach the modified syllabuses to the pupils - 
in the General Certificate of Education classes. 

When the teachers are at work, classroom supervision would 
need to be provided* The principal may* need to give assist- 
ance and guidance. This may relate to how the modified pro- 
gramme of work should be approached and worked on. He may in 
fact be in a position to advise directly on pedagogical and 
methodological aspects. 

Once a teacher continues to work reasonably adequately 
on the basis of the oominuni cation and guidance and supervision 
provided , the principal may need to associate himself with the 
teachers and arrange to evaluate the progress of the work* 

In particular he may need to ask whether the pace of teaching 
is as recommended; whether the practical work and the manner 
of doing it are as specified and other similar questions* in 
addition, principals, with the collaborative assistance of the 
staff, will need to set afoot the necessary machinery for 
teachers to design term tests and similar instruments of 
evaluation of ptipil growth. 
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This will give the principal significant feed-back material 
which he may find useful for the purpose of reporting on the 
programme of work. 

in such reporting the principal himself taking note oi the 
evaluation of performance of the system may indicate the lines 
of new decision making and planning which the programme m 
school would seen; to demand, 

The teacher who has picked up the communication could no* 
proceed to study the situation and such study will entan the 
consideration of such questions as follow; 

What are the features of the revised schemes of work? 

How are they organised? 

How do I use them? 

What particular major principles under ly these schemas? 

Do 1 have the equipment and materials to teach them? 

Whet particular difficulties will my pupils have in 

learning in the modified way? 

How will I help them? Etc,, etc. 

Having recognised the nature of the situation, significant 
problems associated with it and having taken note of the pre- ■ 
ferred lines of approach, the teacher could proceed to plan nis 
lessons, execute them, assist and supervise the pupils at work 
and evaluate his and their progress in appropriate wayfl . 

It will be observed that one of the criti cal features of 
this schematic model is that each person who. picks up a commum 
cation proceeds to amplify it and prepare detailed plans appro- 
priate to his level prior to execution and proceeding with tne 
programme* 
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5 * krai Evaluation , 

with Particular ilelevanoe to Soviet School s 

by S . A * balezin 



At present every nation is living through a period of a 
large-scale scientific and technological revolution. The 
inflow of scientific data is increasing at an unbelievable 
speed, That is why the school and the school teacher are 
facing the problem of deciding on what should not be taught 
rather than on what should be taught to the pupil* In other 
words , the problem of selecting such teaching material that 
would deal with the fundamentals of science on a modern level 



and at the same -time could be within the pupil 1 s comprehen- 
sion, considering his age-group peculiarities, is becoming 
increasingly important . 






Young men and wo men starting ‘ their independent life 
will inevitably become immediate participants in techno* 
logical progress. 



In ^he advanced countries of the world, a considerable 
increase is being observed in the number of workers with 
secondary education, Today it is not infrequent to encounter 
instances where the workers immediately engaged in production 
become research, workers in the full sense of this word. buch 
workers need to have not only skilful hands but also a clear 
creative thinking and effective knowledge. This means that 
the pupils should not only assimilate a certain amount of 
knowledge but also master, to a certain degree, the methods 
used to acquire such knowledge, starting with the simplest 
techniques of research, for instance the qualitative analysis 
of acids and salts./ and up to solving the complex problems 
arising in production processes* 

That is why pupils should, he trained in such a way that 
they could master, to a certain degree, the methods and tech- 
niques used in the jobs they will take up after finishing 
school, to facilitate the solution of the questions on which 
they will nave to take decisions in production and in every- 
day" life. This is possible only on the basis of a certain 
amount of firm knowledge. 

Chemistry plays an outstanding part in the life of the 
contemporary human community. Therefore, chemistry as an. 
instructional subject in the school curriculum is of special 
importance. 
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Form a of current evaluation of th e_pupil's Knowledge i ___oral 
evaluation 



In oral evaluation, the verification of the pupil's 
ledge includes recapitulation and it is one ox the teacher a 
weapons to improve the Knowledge not only oi Jft " res? of 
answering the teacher's question bu „ also tha 
the pupils* * 

Oral evaluation is wlaely used in current evaluation and 
is practised in almost every lesson in chemistry. 1 
as ?oth 8 form of control and instruction thus «prov in| the 
pupil's knowledge and his ability to relate it to the listener. 

Oral evaluation is made before introduction of new material, 
during its presentation and at a special lessor, devoted to 
evaluation, discussion and consolidation oi the pupi b 
ledge , 

• In th® process of oral evaluation, the pupil is asked 

questions on the given topic covering the JlSSi it is 

Also he is given problems to solve* In oral ® 

possible to establish the amount and depth of the knowledge, 

Ks as well as to find out whether the material has 
been assimilated systematically and consciously, 

These are the guiding rules for preparing questions for 
oral evaluation : 

Questions must take into account the amount of material 
envisaged in the syllabus, be brief, simple and easy to unaer-. 
stand, avoiding too much detail, hints and ambiguities, 

Along with the selected particular questions it is also 
advisable to give the pupils small topics lor ^°* SS1 ° n 

(talks) In class VII, for instance (in boviet schools 

Lmiatry as an individual science discipline is i»«oduood^ 
in class VTI) this topic can. be "burning sulphur m xyg , 

£ class VIII, "Chemical properties of chlorine", in class la, 

"Acetic acid% etc. 

The pupil’s talk on the given topic enabies the teacher 
to assess not only the pupil's knowledge but also his ly 
to present the material consistently, systematically and clear y. 

Besides this, pupils are invited to soive problems. This 
serves as an effective means to find out whether the pupil . 
realises the data he has learned, 

Normally, ind. vidual current evaluation cannot be eaeiiy 
combined with the active work of the whole class. Besides, it 
taices time* 
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One of the forms of oral evaluation which can overcome 
this disadvantage is to call to the blackboard two or some- 
times up to four pupils at a time, when each, of them is given 
an individual assignment. While these pupils are thinking 
over their questions or solving the problems * the teacher can 
work with the rest of hhe class. The answers of the pupils 
who are at the blackboard are discussed with the participation 
of the entire class. 

When pupils answer, it ia important for them to be able 
to use prepared charts, tables and various chemical substances. 

Many teachers, parallel with this form of evaluation, 
use a BO-cailed group test talk v^ith the whole class on the 
material studied. This talk takes part the lesson, ox 
sometimes the entire lesson. The teacher puts small questions 
to the class. The pupils give brief answers without leaving 
their seats, and the teach er gives marks to those pupils who 
have been particularly active. 

This evaluation/ talk is sometimes devoted to summing up 
the material of a large section of the syllabus. for instance 
"Interac tion between classes of inorganic compounds or general 
characteristics of* the elements of the halogens group . 



Tentative c r iteria for evaluating the pupil’s kno wledge 

In the U.S.S.R* a five-mark system of evaluation is adop- 
ted, the highest mark being the 5* Marks for one term, half 
year and year are given taking into account the results oi 
current evaluation of the pupil l s knowledge and skills. When, 
final marks are given the virtual level of each pupil’s know- 
ledge and skills should be considered. 

In evaluation of a pupil’s oral answer, the mark 5 is 
given provided the pupil- 

(a) shows understanding' of the chemical essence of studied 
phenomena and processes! correctly formulates chemical con- 
cepts, laws and theories; knows the properties of the studied 
substances and can identify them; correctly reads and makes up 
formulas of substances and conclusions of chemical reactions, 
gives them correct qualitative and quantitative interpre a ion; 

(b) supports the theory he discusses by concrete examples, 
knows the practical significance of chemical laws, theories 
and processes as well as the use of particular substances? 



(c) gives a complete and oonsistant answer displaying 
independent judgment and assimilation not only of the material 
being studied but of the previous material as well; when 
necessary, proceeds from the knowledge learnt when studying 
other curriculum subjects and applied in the prodess of labour 
and vocational aducation. 
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@ mark 4 is given if "the answer ? 

established for the mark 



in the main, meets the 






but the pupil i 



same requiremeii bs 

(a) makes mistakes in accuracy and insignificant mistakes 
in relating facts, definition of concepts, laws and theories 
and in their interpretation while being able to oorrec all 
this easily with the help of the teacher; or 

f'b) gives an an answer only narrating the text of the 
manual; while in replying to the teacher's additional questions 

standing of the material studied- 



displays due unasrsi 



the knowledge and 



The mark p is given if the pupil shows 
understanding of the basic instructional material huts 



(a) experiences 



Lifficulty in supporting, . without the 



teacher * s help, the theory being related, by concrete examples, 
or in explaining the practical significance of chemical laws, 
theories and processes ; 

(b) experiences difficulty in summing up the material and 
drawing conclusions on it, though relates the factual material 
fully enough; 

(c) makes one or two mistakes in reading and writing the 
formulas of substances and conclusions of chemical reactions, 
or in their quantitative and qualitative interpretation; 



or 



(d) gives a schematic answer skipping essential details, 
relating, in the main, the text of the manual while displaying 
insufficient understanding of the individual tenets that he 
freely relates, 

Th© mark 2 is given if the pupil* 

(a) does not kftow, or fails to understand, most of the 
most essential part of the instructional material dealt with 
in the questions put to him; or 

(b) in total, makes mistakes and commits errors indicated 
in the requirements established for the mark 3? or 

(o) even when given the help of the teacher, fails to 
make use, while answering, of the knowledge asaimila e ear . 
in the course of chemistry and in other curriculum 9 utjeet. as 
well as the knowledge and data accumulated m the process ol 
labour and vocational education. 

The mark 1 is given if the pupil shows complete ignorance 
of, and fails to understand, the basic curriculum materials. 

Conclusion 

In Soviet schools, everyday current evaluation of pupils* 
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knowledge and skills forms the basis of evaluation^ pupils' 
knowledge. Use is made of oral individual evaluation, writ ten 
teats, fxperimenta’ tests and experimental problems. 
research that hag been and is being carried out in the U.b.b.R. 
shows that further work is needed to improve the methods of 
verification and evaluation of pupils' knowledge and to achieve 
a better control over the knowledge and skills of pupils as 
well as to achieve more profound knowledge and skills. 
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6 # Report on Evaluation workshop in Chemistry 

Peradeniya, Ceylon, 2-12 August i^bl 

by J#G. Mathews and J. Ratnaike 



Aims of Workshop and this report 

The Workshop wag part of the in-service training of 
Ceylon teachers and other education staff in the construction 
of examinations in chemistry at the Ceylon General Certificate 
of Education, Ordinary Level, The long-term objective is the 
production of a team of question writers and examination con- 
structora who would assist in the design of , O t level examina- 
tions and act as nuclei for further training of teachers. 

This report sets out the sequence of operations during 
the Workshop and the materials which were the outcome of the 
Workshop, In so doing, it is hoped that the report will 
serve as a guide to other countries who wish to conduct 
similar workshops* The scheme of work is not set out as an 
ideal one; it was the first comprehensive workshop to be 
undertaken in this country and those in the future may well 
be modified in the light of this experience; nor are the 
questions which were constructed of equal merit* Neverthe- 
less, a consideration of both the faults' and the virtues of 
this workshop may assist other countries who are revising 
their schemes of examinations in schools* 

PART X 

In preparation for the TE3 Evaluation Workshop the 
following operations were carried throiigh# 

(1) Selection of participants * 

( 2 ) Production of multiple choice and free response test 
items by participants* 

(3) Editing of multiple choice test items by participants, 

( 4 ) Selection of two multiple ohoice pre-te^ts, 

(5) Moderation of multiple choice pre-tests. 

( 6 ) Administration of multiple choice pre-tests for pupils. 

(7) Analysis of data from the administration* 
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(l) Selection of participants 

Three categories of participants were envisaged. 

(a) Supervisory staff as personnel responsible for con- 
trolling and guiding the science programme m the 
field, and as potential in-service leaders for 
future evaluation in-service training courses. 

(b) Controlling examiners, chief examiners and senior 
marking examiners, as personnel directly involve 
in the communication, execution and control of 
examination practices at the tf.C.E. ‘O’, 

(c) Senior teachers who did not belong to category (b) 
but were recommended by the supervisors aw being 
efficient and successful in their teaching. 

The participants in each category were selected according 
to their subject preference, physics, chemistry and biology, 
and across the categories each subject group had an initial 
population of about 25 participants, taking into account the 
possibility of drop-outs before the Workshops, At the Work- 
shop the number expected in each subject was 20. 

(2) Production of multiple ohoioe and free re sponse test items 

by participants 

Each participant was requested to make 40 test items and 
a table of specifications with a content coverage of the whole 
of the G.C.E. *0’ syllabus, and an objective coverage as 
follows* 

15 of I, 10 of II, 10 of III, III’* and 5 of IV. 

The table of specifications portrayed no objectives in the pey- 
chomotor or in the affective domain. 

With the request to make test items the participants were 
also sent a note on the construction of multiple choice test 
items, tables of specif ieations and simple analysis of data 
(see Appendix l). They were also sent a note on the construc- 
tion of structured essay and free response questions (see 
Appendix 2), 

A time schedule of one month was given to the participants 
for this task, with the request that half the items be sent in 
after 15 days. 

When the item sate were received they were identified 
with a code number given to each participant, and the ques- 
tions typed so that the objective suggested by the writer also 
appeared against the question. 



O 




125 



FILMED FROM BEST AVAILABLE COPY 



~ 120 - 



( 3 ) editing of multiple choice teat items by participants 

Each participant was then sent a lull sat of items pro- 
duced by him and by his colleagues and was requested to edit 
these in the following manners 

(i) judge the objective category of the question and 

mark the choice with a circle in the set of objec- 
tive categories indicated at the end of each 
question t 

( i I ) th s»rK the c o j'V a o t choice wi th. a circle for © a c h 
question ; 

(iii) edit questions for obvious errors and ambiguities 
and write the corrections on the particular ques- 
tion. 

These edited seta, vers returned, and formed th© basis of 
the raw pool from which, selection was made for the pre-test. 

( 4 ) Selecti on of, the two multiple choice pre-tests 

The selection was made by two supervisors per subject 
group* Those supervisors were to be the leaders in the 
Evaluation Workshop and had already participated in the opera- 
tions so far. These group loader supervisors then met under 
the guidance of the Mreotor of the Workshop, and selected 
items from the raw pool, making further editing as required. 

They then produced two AO —it Din multiple choice pre-tests. 

( 3 } Mod e ration ,o f_j:jultij> le choice pr e -testjg 

, These selected pr©“ tests were then subjected to moderation 
by the control examiners of the national G.C.E. *Q ? examinations. 
Th©ir function was .mainly to ensure that the pi'e-test had face 
validity when compared with national G . G . E » f U f examinations. 

( 6 ) Administration of multiple choice pre-tes ta for pupils 

The sample to whom the pre-teats were administered came 
from the English medium and consisted of XOU final year G.C.E. 

*0* pupils plus 50 first year G.C.Il- f A 1 pupils, for each of the 
pre-tests. It was necessary to include some pupils from the 
<2*0. E, *A* population because it was recognised that in June 
and July the pupils would not have covered the full syllabus. 

On the other hand, the G.C.E. f A* pupils had not only covered 
the syllabus but had been sufficiently successful in the 
various subjects of the G.C.E. l Q t examination to permit them 
to be in the Advanced Level » It was recognised that 

only tentative extrapolation be made with the data from 

the G.C.E, 1 A 1 population. 

Although translations were made of these pr-e-teats in the 
Sinhala and Tamil languages , apd the tests administered to 
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parallel populations as in the English medium, the data foil w- 
ing from the Sinhala and Tamil media were not intended -o he 
utilised at the Workshop, but were expected to be used w.th 
the participants as follow-up work during the course oi the 
year, after the Workshop, 

(7) Analysis of data from the adminiB tT,atioa M the 

choice -pre-test 

It was recognised that the participants were mainly 
nel involved in classroom evaluation activities an no . . 

who would be taking the primary responsibility for el 

tion design and decision making operations at a national 1 v . 

One of the problems that these participants would meet in 
the analysis of data in a school situation is the smallness 
number in their samples, and the differences between on To 

indices and the indices for their particular situation . - 

highlight this problem, difficulty indices and , J ^total sample 
coefficients were calculated both m terms of the total sample 
population and in terms of the individual school sample popula- 
tion. Further, both the school data and the total samp 

were divided into the G.O.E. and O.O.Jj. ^tand- 

pupil performance characteristics were ref ^ d the 

ard deviations and means calculated for the b.G, . 
q c E ' A ’ sub-sfite, and for the total sample. ihe analysis 
on a school- wise basis was done by the Evaluation Research bmt 
under the direction of the Director of the Workshop, and the 
total sample analyses were done by the supervisors who w«e tc 

act as the group leaders at the Workshop. during imnli 

of this operation, the group leaders were briefed on the impii 
cations of the analysis, and on the apparent trends an m 
p^etations visible from the data. 

The pre-workshop operations in addition to , 

realistic working material for the Workshop, brought oblems 
cipanta to a preliminary level of awareness of the problems 
involved in the construction of multiple choice items, speci- 
fic objectives and content? in the communication tnroug 
tables of specifications} in the preliminary editing pro- 
cesses, including judging the categorising of items accordi s 
to objectives. In addition, the group leaders baa an oppor- 
tunity to involve themselves in the other opera tions^planned 
for the Workshop, such as the selection and rejection of test 
items, and the analysis and interpretation of test data. 



PART II 



The participants at the Workshop 

There were two Directors of the Workshop, one from Gey Ion 
and the other from the United kingdom. There were ten other. 
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participants, in the chemistry group ail of them teachers, 
training college lecturers or education officers in Ceylon, 



The material available 

L Two sets of fixed response questions selected by the 
educational supervisory staff in Ceylon from approximately 
400 such questions submitted during the preliminary work- 
iiach set contained 4'^ pre-tested questions.' 



2 * The data available on each question was* the content 

area and educational objective and the difiiculty index and 
discrimination coefficient as determined by the pre— test* 



The content area indicates the main divisions of the 
material of Ceylon ! 0* level chemistry. 

The educational objectives are divided into five cate- 
gories i 

1. Kecall* 

2 , Comprehension, 

Application (qualitative). 

3 T * Application (quantitative), 

, 4 - Analysis/ synthesis/ evaluation* 

The difficulty index shows the percentage of pre-test 
candidates who answered the question correctly# 

The di s crimination coefficient shows the correlation 
between the performance of candidates on a particular ques- 
tion with t^at in the test as a whole (how the item separates 
good pupils from poor pupils)* 

3* Administrative requirements 5 two rooms, each with f aci li - 
ties for writing; and typing and duplicating staff sufficient 
to type- and duplicate the product of each day’s work during the 
evening. Computing equipment was not required. 

Time available 

There were / working days. The evaluation workshop sess- 
ions amounted to 23 hours# (The participants ware engaged in 
a seminar which included curriculum development work in addi- 
tion to the evaluation workshop*) The participants had an 
average of about 2 hours in each day writing questions or other 
evaluation work in private. 

It was found essential during the operations which follow 
to appoint one participant as secretary* 
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be queue e of operationa 

1* Reviewing objectives . After short preliminary talks by 
the Directors on the aims and methods of the Workshop, parti- 
cipants reviewed the objectives of all the pre-tested questions* 
The latter served the purpose of clarifying to the participants 
the educational objectives in operational terms. 

The first 40 questions were reviewed by general discussion 
under the guidance of one of the Directors, When there was 
divergence of opinion the view of the majority was taken. Where 
divergence of opinion was equal, the lower category of objective 
was selected, subject to the over-riding decision of the Director 
The discussions were lengthy but fruitful. All participants 
agreed that the exercise forced them to consider the fundamental 
objectives of the course in terms of what they should expect 
their students to be able to do at the end of it. 

This sort of discussion requires some skill and experience 
on the part of the Director. In order to give such experience 
to other Ceylon personnel, the Workshop was then divided into 
two groups each in the charge of a Ceylon participant . The two 
groups separately reviewed the educational objectives of the 
remaining 4 c questions. The group© then rejoined and decisions 
were made' on the objectives of questions where they differed 
from the original classification, Finally a revised specifi- 
cation of all 80 items was drawn up. 



2, The next task was to determine* 

the distribution of difficulty indices over the du 
questions ? 

the distribution of discrimination coefficients with 
difficulty indices? 

the distribution of difficulty indices with objectives? 

the distribution of difficulty indices with content 
areas ; 

the distribution of discrimination coefficients with 
content areas? j 



the distribution of discrimination coefficients with 
objectives. 

To expedite the work the participants were divided into 
four groups « Their finding© were then duplicated and given to 

all of them. 



The main findings were* 

2.1 Difficulty increased, with category of objective ( 1/ 11/ III/ 
XIV /IV) * 

2.2 The later the topic- in the course,, the greater the diffi- 
culty. 
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2 * 2 
1*4 



Caloulati on* 
“ions . 



tended to be more difficult than other cues- 



items of medium difficulty seemed to discriminate bes' 



se lection of a 41* -question, test , i'he b‘J questions were 
hen reviewed and classified as follows: 

(i) Acceptable in tha original* 

(ii) Acceptable after minor editing (editing not likely 
to affect the performance in the question) . 

(iii) Acceptable only after major rewriting (rewriting 
likely to affect performance). 

(iv) Rejected. 

The participants &e a group went through the 60 questions 
Several times judging the items according to the following 
criteria: 

jj # 1 Duplication of items . 

5. 2 Too easy or too difficult (some were rejected on these 
grounds . 

3.3 Poor discrimination - Any item with a discrimination 
coefficient below + 0.2 was accepted only if in the sub- 
jective judgment of the participants it was suitable . 

3.4 wo* -functioning responses. 

H s Ambiguity and faui ta in language and content (each parti- 
cipant acted at .reviewer of a part of the questions in 
Ol'CL®!!* "to 3jy<5s<Sd Up t *tiX£ Op© 3 TSl'tXOXI j * 

J.6 Unde six*Ab 1 0 ^fic^cfc on t* Batching* 

The results of this "shredding process" show that of the 
BO questions i 

14 w btg acceptable in the original « 

26 were acceptable after minor editing* 

13 required major rewriting* 

25 were rejected* 

4. haoessarv writing to meet the specifi c ation. f?*?* t the 
remained to determine whether the 4u'aooeptabiejueetions met the 
desired eontant/objective specification. It was . 

content areas wire' oversubscribed, objective 1 was oversubscribed 
and objective 3 * undersubscribed. Therefore five items had to 
be withdrawn and five new items written in their place. The 
latter was done by all the participants working individually, but 
it was found desirable that operation 4 add subsequent operations 
should be undertaken by a small group of three participants plus 
a director. 
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In this way the preferred specif ioation was met approximately 
but not exactly, 

5. Making a random key . It was found that there was some 
imbalance in the correct responses. 

Response (i) in 10 questions. 

Response (ii) in 9 questions. 

Response (iii) in 12> questions. 

Response (iv) in 8 questions. 

The responses in one question, correct response (iii), were 
reordered to give a correct response (iv), thus giving a better 
distribution of responses, 

6. Order of questions , During discussion it was noted that in 
the pre-test the questions were arranged so that the higher objec- 
tives, and more difficult questions, came at the end, it was 
decided to arrange the 40 selected items in such a way as to dis- 
tribute the more difficult items through the test. Two or three 
easier items were deliberately put at the beginning and end of 
the test. 

7. Specification of the test . The essential data on the 40 
questions in the test was assembled. 



Structured and free response questions 

One of the objectives of the workshop waa to give instruc- 
tion and practice in writing questions other tnan those of the 
fixed response type, Participants were asked to write some 
structured and free response questions before coming to the 
Workshop and guidance on how to do. so was sent to them (see 
Appendix 2), 



Not many of these questions were forthcoming, however, and 
most of the writing was done during the second half of the Work- 
ehop. This provided suitable work for those participants who 
were not being used in operations 4-7 abqve. 



It was felt that this part of the Workshop was of at least 
equal importance to the first if desirable effect on teaching 
was to be attended through examinations. 

Time was insufficient to submit the structured and free 
response questions to the same scrutiny given to the fixed res- 
ponse questions ^ 



In conclusion 

One of the most important outcomes of the Workshop was that 
the participants same to realise that teaching and examining were 
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not srparate operations but parts of a larger operation; and 
that akinr part ir a workshop such as this threw a ne ana _ 
clearer' light on the objectives, methods and content ot teacning, 
as well as of examining* 

It was noted by all the participants that a workshop is not 
the best occasion on which to write questions. Ad. eas i tor q»« ' 

tions probably come best when the writer is actively engaged « 
teaching, and much time is required to develop the idea a..d 
refine the question. This workshop was probably at its most 
valuable phase during the discussion, classification ana asssem y 
sessions; and at its least valuable during the question-writing 
sessions * 
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APPENDIX 1 

Intro lu story notes for Evaluation, workshop 
by J. Katnaike 



Introductory note on levels of achievement of p upils j, objectives ) 

In the study of a particular area of content in the syllabus, 
pupils may be taught, and expected, to reach various levels of 
achievement in. the content area. For example, the pupils may 
be able to recall thd reproauce the actual teaching given, and 
no more. This, of course, wii.1 not tell us whether the pupil 
has understood what he is repeating* 

un the other hand, a higher level of achievement would be 
if he it able to apply qualitatively or quantitatively what has 
been taught, in a situation somewhat different to what he has 
met before in the classroom. As teachers, we would then recog- 
nise that the pupil has shown that he has "understood” what has 
been taught. we would call this level "higher" in terms of the 
purposes for which we are teaching, and in the sense that it 
involves more complex operations on the part of the pupil than 
merely recalling* 

lupils may even have achieved a still higher level, of 
being able to recognise problems, select * data, draw general con- 
clusions from data, test hypotheses — in other wo rets use the 
methods of the scientists in his worK* 

These are levels that we as teachers perceive in our d&y-to~ 
day work with pupils. 

These levels pf achievement or objectives g as they are. 
called, have been expressed in various ways* Perhaps the most 
comprehensive classification available today is The Taxonomy of 
Educational Objectives, benjamin b * £ loom el nl . , Longmans Green 
and Go* (1956), Given below is a brief summary of this classi- 
fication. To find out whether the pupil has achieved one or 
other of these levels, we provide him with an opportunity to 
respond at the various levels in other words we ask him ques- 

tions. The questions included as examples below attempt to 
find out at which level of achievement the pupil can. respond. 

SUMMARY OF BLOOM 1 b GbAbiblEIQATXuh OE THE UOttflJLTlVJfi DOMAIN 

L&VtiL 1 , The pupil can provide r esponses involving primarily 
ithGALL of such items as specific facts, terminology , 
conventions, trends, classif ioations, criteria , 
methodology, principles t generalisations « 
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EXAMPLES OF QUESTIONS REQUIRING PUPIL RESPONSES AT LEVEL 1 
Content arsai Air/ combustion* 



The amount , by volume, of carbon dioxide in the atmosphere 
is abouts 

(i) 30 parts per 100 

(ii) 30 parts per 1,000 

(iii) 3 parts per 1,000 

(iv) 2 parts per 10,000 

Content area ; nitrogen peroxide/ equilibrium . 

Which one of the following crystalline substances, when 
heated^strongly in a hard glas test tub©, would NOT give 
brown fumes? 

( i ) Potassium ni irate 

(ii) lead nitrate 

(iii) -Barium nitrate. 

(iv) Copper nitrate 



LEVEL 2 . 



The pupil ca n provide responses involv ing morg_fthan 
simple recall. An element of COMPREHENSION As 
required/ and may be of the types, suc h as the 
followings translation of mathematical verbal 
material into symbolic statements and vice ver sa 5 
interpretation of data and statements iino luding 
simple explanations, summaries, comparisons.) ; 
extra- and in t rapoiation from data , gtatemegtg. x 
( i/e/ extension of trends beyond the giv en data; ; 
dot ersi ination of effects; corollaries # 



EXAMPLES OF QUESTIONS ENQUIRING PUPIL RESPONSES AT LEVEL 2 

Content area * Air/ combustion* 

It is possible to recognise some similarities between the 
burning of common home fuels, and respiration* However, 
it is NOT true to say that, in th© cases of both burning 
and respirations 

(i) the gaseous products are similar in properties. 

(ii) the nib stances burned, and foodstuffs, have at 
east two chemical elements in common. 

(iii) oxygen is required for the reactions to take place, 

(iv) most of the energy from the change appears as 
chemical potential energy of the- products. 

Content area s Nitrogen peroxide/equilibrium 

Nitrogen peroxide eon ‘aining 21^ nitrogen dioxide is 
passed, at one atmosphere pressure, through a long 
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non-porous inert glass tube completely surrounded by 
an apparatus A, The gas emerging at the end of the 
glass tube was found to have a composition of 12 k /» 
nitrogen dioxide* Which one of the folio wing may e 
said of the apparatus A? 



TOLJ, 



(i) A is a heater, 

(ii) A is a cooler, 

(iii) A is a filter for nitrogen dioxide, 

(It) There is insufficient data to identify A. 

The pupil can p rovide responses inv olving the 
APPLICATION of abstrac tions in particular conc rete 
situations.’ " T he abstractions may be in the form, 
of general i deaa , rul es of procedur e, general 
methods^ 'tech nical principles, theo ries. — . RL®. _ 
applications may include nu m erical applicatio ns 
as well. (For convenience we may indicate the 
ieveX of numerical application wi th_ >*■ . ) 



EXAMPLES OF 



QUESTIONS REQUIRING PUPIL RESPONSES AT LEVEL 



5 



Con tent areas Air/ combust! on , 

Which pair of the observations given below, made when 
magnesium was ignited in air. offers the jb.est evidence 
to support the contention that a chamioal change has 
taker, place? 

(i) Heat evolved; colour change. 

; (ii) Colour change; light evolved. 

(iii) night evolved; volume change. 

(iv) Volume change; mass change. 



Content area s Nitrogen peroxide/equilibrium. 

When a mixture of chlorine and helium ( called gas mixture 
X) is in a container at room temperature and 1 atmosphere 
pressure is passed through a series of long inert parous 
tubes, it is observed that the emerging gas has a relative 
density different to that of the original ^gas . Which 
one of the following (and for what reason) would you 
"expect when instead of gas X, nitrogen peroxide is passed 
through the above set-up, under the same conditions of 
temperature and pressure? 

(i) i\o change in the relative density because gas 
mixture a is a mixture of lighter and heavier 
components whereas nitrogen peroxide is a 
substance. 



(ii) No change in the relative density because gas 
mixture X is a mixture of lighter and heavier 
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